The Autonomy Ratio: A Better Way to Evaluate AI Video Tools

Every AI video tool is being marketed the same way. Faster. Higher fidelity. More realistic motion. The demo reels are getting harder to tell apart. The benchmarks compare models on shot quality and prompt adherence, which are real things to measure but increasingly the wrong things to optimize for.

The question that actually matters is different. When you finish a project with the tool, how much of the work did the tool actually do, and how much did you do for it.

We call this the autonomy ratio. It is the most useful single number we have found for describing what an AI video system is actually doing. This post explains what it is, how to measure it, and why every team buying AI video tools should start asking for it.

What the autonomy ratio measures

The autonomy ratio has two parts, and both matter.

The first is creative decisions. Which take captures the moment. What pacing the cut should have. Which song fits the brand. Whether the second character's energy is right. These are taste calls. The user is paying to express taste. A good system does not take these decisions away from the human, it gives the human a clean way to make them.

The second is execution clicks. Once a decision is made, how much manual work does it take to land in the timeline. Trimming each cut frame by frame. Dragging clips into position. Re-rendering when something changes. Lining up audio. Applying the same color treatment to twelve scenes. None of this is creative work. It is labor.

The autonomy ratio is the percentage of total work the system absorbs on its own, weighted across both. A system that takes the creative decisions away from the user has a high autonomy ratio on paper but is the wrong product. A system that lets the user make every creative call but still demands hundreds of manual clicks to execute those calls is also wrong, just in the other direction. The right system pushes execution toward fully automated while keeping creative decisions firmly in the user's hands.

A traditional NLE workflow has an autonomy ratio near zero. The user makes every creative decision and also performs every click to execute it.

A pure prompt-to-video tool can look like it has a high ratio on a single shot, because the model executes the prompt with no clicks. But it has also taken the creative decisions away from the user, which is not the same thing as helping the user. And on a multi-shot project the ratio collapses anyway, because the tool cannot coordinate across shots and the user ends up doing the cross-shot work by hand outside the tool.

Real video editing systems live in a specific middle. The user makes the creative calls. The system absorbs the execution.

Why a single number is more useful than benchmarks

Shot-quality benchmarks measure how good a single generation looks. They do not measure whether the system can finish a project. The autonomy ratio does, because it forces you to count every decision the user had to make and every click the user had to perform to land it.

Two tools can produce identical-looking final videos with completely different autonomy ratios. One had the user clicking 200 times to get there. The other had the user clicking 12 times to land the same set of creative choices. The user-facing experience is not the same product, and yet shot-quality benchmarks would call them equivalent.

The autonomy ratio is also harder to game. You cannot cherry-pick a flattering prompt for it. The number is computed across a real project, end to end, with all the boring intermediate clicks counted.

How to measure it

There are two things to count.

Creative decisions. Any choice that affects the final output that the user explicitly wants to make. Take selection. Pacing. Music. Visual style. Brand calls. The system should make this set as small as possible by handling everything that is not a taste call, and as clean as possible by surfacing each decision in a single touchpoint instead of scattered across a timeline.

Execution clicks. Every manual interaction needed to translate decisions into the final timeline. Cuts placed. Trims adjusted. Layers aligned. Effects applied. Re-renders triggered. The system should drive this number as close to zero as possible.

The ratio is then: of the total work in the project, what percent did the system handle without human input. Decisions the user explicitly chose to make are not counted against the ratio, because the user wanted to make them. Clicks the system forced the user to perform to execute those decisions are counted against it, because they should not have been required.

The instrumentation is straightforward inside a single tool. It is harder to compare across tools, because every system structures its decision space and its UI differently. The most honest comparison today is within a single project type, on a single tool, over time. Improvement in the ratio is the signal. The industry will eventually need a shared standard for cross-tool comparison. Somebody will need to write it.

What the numbers actually look like

To make this concrete, here is what we see internally at Poolday. We have been tracking the autonomy ratio across projects for months.

On straightforward edits, talking-head trim-and-publish, B-roll insertion, social cutdowns from a master, the ratio is at one hundred percent. The user makes the creative calls at the front of the project, and the system finishes the rest without further input or clicks.

On more complex multi-character, multi-scene work with brand constraints and creative requirements, the ratio sits at sixty-four percent. The remaining thirty-six percent is split. Some of it is creative decisions the user wants to keep, and that is healthy. The rest is execution clicks the user is still being forced to perform, and that is the part we are eliminating.

That gap is the work. Pushing sixty-four toward one hundred means giving the user more leverage on each creative call while removing the manual labor between calls. The creative input stays. The labor goes.

Why teams should start asking for this number

If you are a buyer of AI video tools, ask vendors two questions on a project type that matches yours. How many creative decisions does your user make to finish a project. How many clicks do they make to execute those decisions. A vendor who cannot answer either has not measured the workflow. A vendor who answers the first with a low number is taking creative control away from your team, which is usually the wrong tradeoff. A vendor who answers the second with a high number is selling you software that still expects you to do the work.

If you are a builder, instrument both. The metric forces you to separate the two kinds of work and optimize each one differently. Creative decisions should be made fewer in number and richer in leverage. Execution clicks should approach zero. Confusing the two is the most common architectural mistake in this category, and it shows up immediately when you start counting.

The short version

Shot quality is a feature. Autonomy is the product, and autonomy means two things at once: protecting the user's creative decisions and absorbing the clicks required to execute them. The autonomy ratio is the cleanest single number we have found for measuring whether an AI video tool does both, and we think the entire category will eventually be evaluated this way.