15

I Had to See the Shot Before I Knew What to Change

The desired effect can be clear while the way to film it remains unresolved. Building one opening shot made me reconsider how much a creative tool should ask upfront.

An oil painting of a filmmaker studying a projected scene beside storyboard sketches

While building mstudio, I keep returning to a practical question: how clearly should someone have to describe a film before they can start making it?

mstudio is a desktop video studio I am developing. Agents can write scripts, design shots, and generate material; people can also edit directly on the canvas and timeline. Using it for my own video work, I found that some requirements were clear from the beginning. The product had a particular shape. Certain details had to stay accurate. Other requirements became easier to express only after I saw an image.

A shot that the written plan had not resolved

In a video about a lunch bag, one shot was supposed to put the viewer inside the bag. As the lid opened, light would enter from outside.

That viewing effect was clear. The camera position, which side of the lid would be visible, and how the opening action would appear still needed to be tried. Writing those details into a plan did not establish that they would work together.

During production, the agent adapted to a reference image in which the bag was already open. It replaced the opening action with a static display. After the action was restored, the generated image had two lids. Further inspection exposed incompatible viewpoints and actions in the prompt.

At that point, there was something specific to revise: keep the experience of looking out from inside the bag, and reconsider the camera position, occlusion, and reference image.

If a form had asked me to specify all those details at the beginning, I might still have supplied the wrong combination. Nor should a camera position or duration proposed by the assistant become an unchangeable requirement simply because I liked the creative idea.

Accepting a proposal can mean that a direction is worth trying. It does not necessarily mean that every implementation detail in the proposal has been settled.

Moving forward must leave a way back

This affects the structure of the product. Treat scripts, storyboards, and generation as successive approval gates, and each earlier stage starts to feel closed once the next one begins. Actual production goes back. A failed video may need a different reference or a revised shot. A storyboard may reveal that a sentence in the script cannot be communicated by the proposed composition.

I have therefore kept scripts, the canvas, and the timeline editable. They expose different problems. The script makes story and sound readable. A storyboard exposes composition, object positions, and a particular moment. The timeline reveals whether an action finishes and whether a cut arrives too early. One block of text is a poor substitute for all three.

These surfaces do not have to become additional approval work. Changing one clip should not require approving the whole script again. A person can point to the relevant object and continue the conversation, or make a direct adjustment.

Some questions are still worth asking early: product facts, required content, and constraints that must not change. Getting those right prevents avoidable work. Composition and pacing, however, often become easier to discuss after there is something concrete to inspect.

Make room for revision without excusing errors

It would be convenient to describe every failure as helping a person discover what they want. That would also be wrong. An inaccurate product, an unfinished action, or software that adds conflicting text to a prompt is still a problem to fix. Repeatedly asking people to correct the software is not the creative process I want to preserve.

The revision I want to support is more specific: someone sees a direction, chooses what to keep, and changes another part. The intended viewing effect can survive while the way to achieve it changes.

This is also why direct editing still matters. If a cut is early, dragging it may be enough. If a shot has an unresolved spatial relationship, referencing it for an agent to reconsider may be more useful. The task should determine the interaction.

The product needs more than a conversation transcript to support this. People need to return to the particular storyboard, generation task, or clip they mean. The assistant needs to distinguish a tentative experiment from a requirement that should govern later work.

mstudio does not yet handle all of these returns smoothly. Generation and scoped editing still produce problems. But I no longer want the starting condition to be a perfectly specified film. I want someone to be able to look at a result, say which part they would change, and continue from there.