A camera roll and a prompt go in; a printed book comes out. The system splits the work in two: an LLM decides everything that is taste — theme, story, pacing, even the layout schemas — and deterministic code computes everything that is geometry — rectangles, crops, DPI. This page walks the five stages and the one principle that holds them together.
01The wedge
10–20 h
to build one photo book by hand today — the reviewer roundups' consistent number
#1
abandonment cause: "too many photos" → decision paralysis; autofill is widely panned, users rebuild manually
Nobody owns "talk to your photos on the web, get a book." One honesty check rides along: the incumbents' moats are physical — color-accurate print, layflat pages, reliable shipping. The AI wins the first twenty minutes; fulfillment wins the repeat order.
→ reference.md §1
02The one principle
The LLM owns intent — up to and including authoring the book's design system. Deterministic code owns per-photo geometry. Everything else in the architecture is this sentence, applied. Only intent crosses the seam — which photos, how important, what story. Every geometry number — rectangles, crop boxes, DPI — is computed after, and never by the model.
The point of the split: four promises an LLM redrawing a canvas can never make, and a deterministic builder can prove —
every photo placed ✓≥300 DPI at placed size ✓faces never cut ✓exact bleed ✓
Determinism has a quieter payoff too: unchanged inputs rebuild byte-identically — which is what makes safe conversational editing possible at all (§08).
→ architecture.md §2 · integration.md §1
03Five stages
1 · ingest
previews first
~1024px previews upload first, all analysis runs on them; full-res trickles up behind. EXIF / HEIC / clock-skew normalized.
fill → crop → verify → k valid candidates, a VLM picks the nicest.
→
5 · render + edit
HTML → print
existing Artifex path; edits are blueprint diffs against pinned spreads.
The ordering rule underneath all of it
Cheap, fast, deterministic operations run early; intelligent, slow, costly calls run late — consuming the structure the cheap ops built. Hard-rejects happen only at the cheap end; the expensive models only ever see what survived.
millisecond CPU gatesdeep models (embeddings, faces)batched VLMone frontier call
→ architecture.md §3–5
04Photo intelligence — roll in, manifest out
Turn 800 raw photos into a structured object the book brain can reason over: facts and scores, zero design. The shape is a funnel — each stage pays only for what survived the last.
uploaded800
after technical cull640
after dedup + burst collapse430
keepers, grouped into events96
placed in the book~40
Numbers illustrative — the shape is the point. Only the ~40 placed photos ever need their full-res originals (at crop time).
Collapse before you count.
Dedup runs before ranking and before event weight: ten copies of one sunset must not buy three spreads.
Boundaries from rhythm, not thresholds.
An event boundary fires when a time gap is ≥17× the local geometric mean — so a wedding and a road trip both segment right. GPS velocity splits ceremony-vs-reception.
Faces become people.
Face clusters ranked by frequency = key people: protected from culling, cover-eligible, and the only source a caption may take a name from.
Adaptive segmentation: the threshold is relative to the local capture rhythm, so bursty and sparse albums both work. Dirty timestamps are the silent killer — multi-device clock skew is corrected first, and EXIF-stripped photos (WhatsApp, scans) go to an "undated" tray rather than being interleaved plausibly-but-wrong.
Saliency — computed once, spent everywhere.One "where the eye goes" map (the DINOv2 attention map, a free by-product of dedup) is reused down the whole pipeline: the blur gate counts only sharp∩salient tiles (a sharp watermark saves nothing); the palette weighs pixels by it so the subject's colors set the theme; cover picking hunts the largest low-saliency rectangle to park the title; and at build time a dedicated model (U²-Net class) feeds the crop's importance map. One rule everywhere: saliency is soft heat — faces ride on top as hard constraints, never just brighter pixels.
Captions print permanently.So they are grounded, not guessed: names only from face clusters + user tags; places and dates only from EXIF; every caption carries a confidence field; the cover and any person-naming caption get human review. Impersonal AI captions are the market's #1 quality complaint — grounding is the answer, not longer prompts.
The MVP runs this stage VLM-heavy: clustering (adaptive time-gap) and the per-chapter OKLCH palette stay deterministic, while cull & insights come from batched VLM calls with structured outputs — one prompt to iterate instead of ~15 thresholds across 8 models, accepting the latency and cost to prove the architecture. The classic CV stack (blur ∩ saliency, exposure/noise, pHash + DINO dedup, keeper-per-burst, NIMA) arrives with the "deterministicate" step (§10). Either variant emits the same manifest.
→ pipeline.md §1–2 · roadmap.svg
05Book brain — one call, everything frozen
One frontier call reads the manifest plus the user's prompt and emits the Blueprint: theme tokens, 4–8 spread templates authored for this book, the per-spread photo plan (ids, importance, roles — no coordinates), pacing, narrative arc, titles and caption intents. Authored templates pass a deterministic linter (validity only, zero taste); lint failure falls back to a stock hand-built library — a fixed library alone would reproduce exactly the "autofill" look the market pans.
Pacing is what separates a designed book from autofill — and it's blueprint-owned: the brain cuts the keeper stream into spreads and sets each spread's photo count. Hero cadence, density curve, breathing room. The builder never interprets pacing; it arrives compiled into the per-spread plan.
Why one call, not several.
Blueprint + narrative as separate whole-book calls tell overlapping stories that can disagree — and pay for the book context twice.
Why freeze before building.
With theme + templates frozen, every spread builds in parallel with zero drift. Parallel LLM rendering without a freeze is a documented drift failure elsewhere.
→ pipeline.md §3 · architecture.md §4.3
06Build — two engines, one crop rule, then taste
Per spread, in parallel, against the frozen blueprint: pick an engine → fill slots → crop → verify → emit candidates. The engine choice is the only branch.
School A — templates + Hungarian assignment
A template declares slots (rectangle, target aspect, importance capacity). The real question: which photo goes in which slot? Score every photo–slot pair, then solve for the globally cheapest assignment — not greedily. Greedy is locally sensible and globally wrong, because slots are a shared resource. Watch what greedy does to P2:
greedy pickHungarian (optimal)
GREEDY total cost
HUNGARIAN total cost
Hungarian / Jonker–Volgenant is O(n³) — microseconds at ≤8 slots — and exactly optimal while the objective stays a sum of independent pair costs. The importance × slot-area term makes greedy's trap routine: a filler cheaply claims the hero slot and forces the hero into a small one.
School B — BRIC, a slicing tree that never crops
No template library. A layout is a binary tree: leaves are photos, each interior node cuts its box in two — STACK (heights add) or ROW (inverse aspects add, "resistors in parallel"). Every photo keeps its native aspect; you search over tree shapes for one whose root aspect matches the page. Same three photos, two trees, two valid layouts, zero crop:
Rendered rectangles — solved live via the gutter-exact affine pass. Each label shows the preserved aspect.The slicing tree that produced it. Node label = the cut; leaves are the photos.
For a two-page spread, the root is forced to be a ROW cut at the spine so no photo straddles the binding. Interlocking pinwheel/brick layouts (no slicing tree) ship as School-A macro-templates — Hungarian fills them, no third engine.
Face-safe crop — a hard constraint, not a preference
The slot has a fixed aspect; the photo usually doesn't; so you crop. The naive move — center on the saliency mass (eyes, torso) — quietly slices foreheads off: the cranium is low-saliency, so a short window trades it away for chest. The fix isn't reweighting, it's a feasibility filter: expand each face to a head box and hard-reject any crop that doesn't contain it entirely. Head-cutting crops are never even scored.
drag toward wide to force the head-cut case
SOURCE PHOTO · both crop windows drawn on it
naive (saliency-centred) head-safe · dashed box = expanded head box that must stay inside.
WHAT THE SLOT ACTUALLY SHOWS
naive → head cuthead-safe ✓
When heads are too spread for any window of the slot's aspect, containment is infeasible → a fallback ladder: relax tightness → demote a bystander head → ask the layout for a differently-shaped slot (best fix: change the layout, not the photo) → letterbox → full-page slot.
Then: generate cheap, choose with taste
The builder is microseconds-cheap, so emit k = 3–5 valid candidates per spread — different templates, trees, crop anchors, every one already past the hard gates — render thumbnails, and let a VLM pick the nicest. Correctness stays deterministic; aesthetics become learned; zero training. This one trick subsumes most of the practical value of the 2024–26 learned-layout literature.
→ pipeline.md §4 · reference.md §4
07Six artifacts, one direction
One config precedes the pipeline; five named objects cross the stage seams; everything else is stage-internal. All of them are content-addressed (input hashes) — so diffs, pins and caching all reduce to hash comparisons.
t=0 · user + catalog PrintSpec
physical truth + quality floor — read-only to the LLM
→
photo intel → brain Photo manifest
facts + scores, zero design
→
brain → build Blueprint
all intent, zero geometry
→
cache → build (joins in) PhotoAsset
pixel facts by photo_id — never crosses the LLM
→
build → render SpreadLayout
exact geometry, with proof: constraints_met
→
render → storage / print ArtworkEntry
the existing Artifex page shape (BODY:N)
crop_src_px is always in full-resolution source pixels, so print pulls native 300 DPI — this is the architectural flip from today, where user photos enter generation as ≤100px style references with no placement guarantee.
The one backward edge.Every photo carries a hard print ceiling (min(w,h)/dpi) and a soft floor from its role. A hero too low-res to print big is ceiling < floor — the spread comes back infeasible, and the partition is revised (move, demote, or drop). Bounds flow forward; only hard failures flow back.
→ integration.md §2 · pipeline.md §4.5
08The edit loop — pins, diffs, idempotence
User actions (swap a photo, keep this spread, fix a caption) become pins — constraints the book brain must honor on every re-run. Chat edits apply as blueprint diffs, and the spread is the diff unit: "more of grandma" never reshuffles page 3. This only works because the builder is deterministic — unchanged inputs rebuild byte-identically, so an untouched spread provably cannot change.
One chat edit, six spreads: the pinned spread is immutable to the book brain; only the two spreads whose inputs changed rebuild. The rest are hash-equal — not "hopefully similar," identical.
"swap this photo"
builder only
burst / near-dup siblings offered first — the manifest kept the refs
"re-roll this spread"
builder only
surfaces the best-of-k runners-up — near-free, they already exist
"warmer captions"
caption call only
layouts untouched
"more of grandma"
partition + affected spreads
re-plan, rebuild only what changed
"make it moodier"
tokens only
the design system re-skins; geometry untouched
"make it 28 pages"
PrintSpec edit
the one user-mutable field — and the cover re-derives, because spine width = f(page count)
→ pipeline.md §5 · integration.md §6
09What "good" means — and how it's checked
Hard gates — automated, build-blocking.
Read straight off constraints_met: every photo placed · zero head cuts · ≥300 DPI at placed size · bleed coverage · no face in the spine band · captions grounded, never invented. A failure blocks the build; it is not a score.
Soft quality — a VLM judge, fixed rubric.
Pacing/rhythm, whitespace balance, crop quality, caption groundedness, theme coherence — per spread and per book, plus pairwise A/B between pipeline versions. The same rubric doubles as the best-of-k selector.
Both are exercised against 3–5 labeled golden albums, deliberately including the messy real world: multi-device clock skew, WhatsApp-stripped EXIF, scans, one 2000-photo dump. They're also the only honest way to calibrate the pipeline's ~15 tune-per-dataset thresholds.
→ pipeline.md §6 · architecture.md §7
10Phasing — VLM-heavy first, then deterministicate
The MVP deliberately runs VLM-heavy — high latency and cost accepted; the focus is architecture and upgradability. Then determinism is bought where it pays, and everything else lands as improvements.
P1 · MVPProve the loop
simple upload · adaptive time-gap clustering · per-chapter OKLCH palette · cull & insights via batched, structured VLM calls · book brain: chapter-local narrative, dynamic pacing, a few layouts built on the palette · Hungarian assignment over pre-built layouts, centered crop