Did the chained workflow keep the character?
Castbook chains open-weights models (keyframe → video → stitch) on Modal GPUs, scores every shot against the character's face, re-rolls the shots that drift, and records what each step cost. Workflows are versioned configs, so two recipes can be run on the same script and compared on identity, retries and dollars per accepted second of video.
Character c01: one generated, front-facing portrait (no real people). Every score below is
ArcFace similarity to this face.
Same script, same character, three ways to chain it
A 3-shot script (a street at dawn, a café counter, a café window) through three workflow versions. Each shot's clip must pass the identity gate: p10 similarity ≥ 0.5 over the frames that can be scored.
keyframe_i2v@2 · 2/3 shots passed the identity gatekeyframe_flf@1 · 2/3 shots passed the identity gatelastframe_chain@1 · 0/3 shots passed the identity gate| Workflow | Accepted | First try | Min p10 | Attempts | Warm $ | $ / accepted s |
|---|---|---|---|---|---|---|
keyframe_i2v@2 | 2/3 | 1/3 | 0.50 | 10 | $0.129 | $0.0127 |
keyframe_flf@1 | 2/3 | 2/3 | 0.52 | 12 | $0.117 | $0.0116 |
lastframe_chain@1 | 0/3 | 0/3 | – | 9 | $0.142 | – |
Chaining off the last frame can't change scenes. Shots 2 and 3 never reach the café: in the "café window" shot she's still on the street. Its faces stay too small to verify, so nothing ships. Re-anchoring with start and end keyframes beat the usual keyframe → video recipe: both verifiable shots passed on the first try, identity was higher, and cost per accepted second was 9% lower even with an extra keyframe per shot, because no clip had to be re-rolled. One character and one seed, so read it as a direction, not a result. Full scoreboard →
A number you can trust, and knowing when you can't
The first run scored every frame and got a clip mean of 0.33, min 0.01: it looked like the face drifted. It mostly hadn't. She turns toward a window, and ArcFace collapses on profiles even for the right person. The judge now leaves out frames turned past 45°, frames with a hand over the face, and faces under 64 px, and returns pass, fail or unverifiable instead of a misleading low score.
Similarity per sampled frame. Shaded frames are profile and not scored; diamonds mark a hand on the face. Hover for values, click to seek. Run page →
Fix it at the cheap step
Shot 1's face was 79 px wide in the keyframe, about 53 px in the 480p clip, so every frame was unverifiable. Rendering clips at 720p made it measurable, but at 3.6× the clip cost. The framing check predicts the clip's face size from the keyframe and re-frames the keyframe instead, for about $0.003.
| Workflow | Accepted | Warm $ | $ / accepted s |
|---|---|---|---|
keyframe_flf@2 | 2/3 | $0.120 | $0.0118 |
keyframe_flf@3 | 2/3 | $0.345 | $0.0341 |
keyframe_flf@4 | 2/3 | $0.122 | $0.0121 |
Shot 1 in each: @2 unverifiable (0% scorable),
@3 fail at p10 0.44,
@4 fail at p10 0.48, the same answer as 720p
with a 480p clip (3.6× cheaper).
The answer is honest: once the face is big enough to measure, this shot does drift,
so lowering the threshold would have shipped a bad clip.
Full scoreboard →
How it works
- Character: one portrait → ArcFace embedding.
- Keyframes: Qwen-Image-Edit-2511 composes each shot's start (and end) still from the portrait. Gated: identity, plus a face-size and camera-direction check, re-rolled or re-prompted by cause.
- Clips: Wan 2.2 I2V A14B animates between keyframes. Gated the same way.
- Stitch: ffmpeg concat. One failed shot degrades the run; it doesn't fail it.
- Every step is content-addressed:
sha256(step, model@rev, params, inputs, seed). A re-run or a new workflow version only computes what changed, so A/B arms share upstream work. - On Modal: one GPU class per model family, weights on a Volume, a CPU orchestrator and a FastAPI layer that scale on their own. The unit tests run in CI without a GPU.
POST /v1/workflows/keyframe_flf@4/runs
Idempotency-Key: cafe-morning-c01-1
202 {
"request_id": "req_6e1e633dc910",
"status": "queued",
"replayed": false,
"estimate": { "usd_low": 0.081, "usd_high": 1.267 },
"status_url": "…/v1/runs/req_6e1e633dc910",
"cancel_url": "…/v1/runs/req_6e1e633dc910/cancel",
"result_url": "…/v1/runs/req_6e1e633dc910/result"
}
Async submit, poll and cancel; an Idempotency-Key replays the original run instead
of paying twice; the run reports per-shot attempts, verdicts and billed GPU time. The estimate is priced
from measured timings before anything runs (here for 3 shots: best case vs every retry and a cold
start).
What a run costs
| Step | Model | Cold load | Inference | $ (cold) | $ (warm) |
|---|---|---|---|---|---|
| Keyframe | Qwen-Image-Edit-2511, 4-step Lightning | 57 s | 5.4 s | $0.068 | $0.006 |
| Clip | Wan 2.2 I2V A14B, 4-step Lightning | 227 s | 22.4 s | $0.274 | $0.025 |
Warm inference is already cheap and fast; the cost is the cold start. Wan's weights took 227 s to load and cast in this run, so most of a small run's bill is model loading. Re-saved bf16 weights plus Modal memory snapshots are the next step.
What's next
- Six shots, one API call. A 30-second script through the API to a stitched video and a run page.
- Retry ladder as config: same → degrade → alternate model → fallback shot, so an agent's retry rules become testable code with bounded cost.
- Calibration: fit the identity threshold to face size on hand-labeled frames, and use the gate to measure how often an LLM critic agrees with it.
- Cold starts: bf16 weights and memory snapshots, with a before/after table.
- More arms: full-step vs 4-step Wan, and hosted video APIs behind the same adapter interface.