How we made an AI music video
One song, one family, sixteen shots and a 2:19 film — with no camera, because every frame of it was generated. This is the whole process in the order we did it: how one family survived sixteen separate generations, which six moments were worth animating, how sixteen shots became thirty cuts, and exactly what the whole thing cost in tokens.
The four decisions that made it work
1 A place, not a person
Pick one location or object that repeats — a doorframe, a kitchen table, a front step. It carries continuity far more reliably than a face, and it gives every shot a reason to belong.
2 Two references, and keep them
The editor gives you two reference slots per generation — one to carry the people, one to carry the place. Two is enough. The mistake is churn: a different pair every time walks the face somewhere new by shot six.
3 Stills plus a little motion
We animated six of sixteen shots. Motion is the expensive part, so spend it on the moments that carry the song — not evenly across the film.
4 Leave room at the edges
Image models top out at 2:3, and a 9:16 film crops roughly 8% off each side of that. Write the prompt so the subject sits away from the edges — a face pushed to one side gets trimmed, and you only find out at render.
Most "AI video" ends up as a slideshow with music behind it, because the hard part isn't generating a picture — it's making sixteen pictures feel like they belong to the same film. Faces drift between generations. Rooms change colour. The song and the cuts pull against each other. So we made one properly and wrote down what worked: a film called "Hold Still", about a kitchen doorframe with pencil height marks on it, and a mother measuring her daughter against it as she grows. What holds it together is that the through-line is a place, not a face: the doorframe is in almost every shot, so when a face shifts between generations you never lose the thread. Everything below is reproducible in the editor with the same buttons we used.
Perfect for
Some moments this fits — there are more.
A song you wrote
You have the words; this is how they become a film.
An anniversary
A story told across years, without needing photos from all of them.
A tribute
For people and moments there is no footage of.
A short film idea
Test a concept end to end before you book anyone or anything.
A songwriter's first video
Bring the lyrics; the song gets written and sung here, then cut to.
A portfolio piece
A finished 9:16 or 16:9 film with no OpenMuse watermark and commercially licensed included music.
Learning the editor
One real project teaches the whole tool faster than any tour.
Tips for the best result
A few small things make a noticeable difference.
- 1
Lock your two references
You attach two reference images per generation. Use the same two across a scene — one for the cast, one for the room — and swap them only when the room actually changes. Faces drift when the references drift.
- 2
Name who is wearing what
If two people are in frame, "she is in a green t-shirt" is ambiguous and the model will guess — ours dressed the mother instead of the daughter. Say "the girl", "the older woman", every time.
- 3
Check it at phone size
Our whole film hinged on pencil marks on a doorframe, and at the size people actually watch, they vanished. The fix was one close-up shot of the marks, not bolder marks everywhere. Shrink your frames to phone size before you commit to a look.
- 4
The song is the clock
Let the vocal decide the cut. Our final shot holds on the last sung word for three and a half seconds, and it is the only reason the ending lands.
- 5
A refusal is usually the reference, not the words
Content filters read your reference images, not just your words. A shot that gets rejected as a reference may be perfectly fine on its own — swap the reference before you start rewriting the prompt.
How it works
- 1
Start with the song
Write the lyrics yourself or have them written for you, then generate the track — 30 tokens. Do this first, not last: the song's length and where its chorus falls decide how many shots you need and where the film should turn. Ours came back at 2:19 with a breakdown at 1:27, and that shaped everything after.
- 2
Generate your cast and your place
Make one establishing image you like, at the shape you intend to finish in. Then generate every following shot with earlier shots attached as references, so the same people stay in the same room. You get two slots, so choose deliberately — one that holds the people, one that holds the place — and change them only when the scene does.
- 3
Animate only what needs to move
Bring a still to life as a short clip where the song asks for it — the opening, the emotional peak, the moment before the last chorus. We animated six of sixteen. Video is priced by the second, so this is the line item to be deliberate about.
- 4
Arrange, then render
Add everything to the timeline and press Auto-arrange: it fits your shots to the length of your song and gives every clip a camera move and a transition, in whichever look you pick. Where the song runs longer than your material it comes back to shots rather than holding them — which is how sixteen shots became thirty cuts across 2:19. Then fine-tune anything by hand, and render in the shape you picked back at step two.
Frequently asked questions
Other ways to make it
Make a travel video with AI
A 2:44 road-trip film, one couple, fifteen shots — and nobody left the house.
Read moreLyric video
Your photo becomes an album cover the song sings over.
Read moreNeon ring video
Your photo in a neon ring that pulses with your song.
Read moreAnimate old photos
Add gentle motion and music to a single still.
Read more