How we know

Words we use

The words the stories and the lab notes use, in plain terms. Each term links to the place in the repository that explains it.

Request

One step of the main model.

Claude gets the whole conversation so far, decides on one thing to do (read a file, run a search, make an edit), the tool answers, and the answer joins the conversation. Then the whole conversation goes back to the model for the next step. The lab counts requests first and tokens second.

Context

Everything the conversation holds, re-read by every request.

The system prompt, the tools, your messages, every file Claude read and every tool result. It grows the longer you work, and each request re-reads all of it. That re-reading is most of what a session costs.

Cache read

Re-reading context that is already cached: cheap per token, not free.

On Opus 5.5 a cached token costs $0.20 per million instead of $4. In a session holding 220,000 tokens, every step costs about $0.044 before Claude has thought about anything.

Cache write and its lifetime

The first time text is cached it is written, at a higher price, and it expires when idle.

Claude Code writes with the one-hour lifetime: $8 per million tokens on Opus 5.5 (a five-minute write is $5). An idle session's cache expires after 1 hour on a subscription, 5 minutes in overage or on an API key; the next prompt then writes the whole context again.

Carry

What a step leaves in the context, re-read by every later request.

Whatever Claude reads stays in the context until a compaction removes it, and every later request reads it again. In the author's interactive sessions a token added to the conversation was re-read by a median of 143 later requests: on Opus 5.5 about $37 per million tokens over its life, against $8 to write it. So what a step leaves behind can cost more than the step itself. The lab's benchmark adds a carry column: the tokens a run added × 140 later requests × the cache-read price.

Compaction

Claude Code replacing the conversation with a summary.

It makes room and makes later requests cheaper, but detail is lost to the summary. smart-compact can compact an idle session while its cache is still warm, when that costs less than keeping it warm.

Fresh and long session

A new conversation, or one that has already read the project.

The lab's long session forks a prepared session that has already read all of click's source, about 220k tokens of context. That is where real work spends most of its time, and where a removed step removes the most re-reading.

Plain Claude Code

Claude Code with none of these mods: the baseline.

Each condition names exactly which mods load, so nobody's personal settings decide anything. Lab notes 8 tells how a plain baseline once was not plain.

Hand-off

Claude giving a self-contained task to a cheaper worker and checking the result.

Claude calls delegate's hand_off tool with a task that stands alone, its kind (read, mechanical, build), the places it touches and a check command. It pays on big, specified work (a feature, tests); on a lookup it costs more, because a hand-off takes two to five requests of its own.

Worker

The cheaper model that does handed-off work, in its own small context.

Haiku at low effort for read and mechanical work, Sonnet at medium for build work. Every request of the main model re-reads its whole context; a worker reads only its own.

Verified

The one line delegate puts first: the worker's work checked by the tool itself.

The worker ends its report with a VERIFY block: the command that runs only its new tests, and what it left open. delegate runs it beside the check and counts both. Verified lets Claude report without running anything again; Not verified lists each reason.

Order and advice

An order in the person's name is followed; advice from a tool almost never is.

A note that told Claude a faster way was acted on 0 times in 45. An order phrased as the person's setting was followed every time. So the pack's mods either answer inside what Claude already reads, or give an order where the person's own action (approving a plan, a setting) stands behind it.

Cell

A short program that makes several real tool calls as one step.

cells' run_cell runs JavaScript whose calls are real Read, Edit, Write, Bash and LSP calls, with the usual permissions: several edits and their check in one request instead of four, and every edit still shows as a diff that /rewind can undo.

Language server

The engine your editor uses to know where a name is defined and used.

Unlike grep, it tells a call from a comment or a namesake. lsp-plus asks it inside the search Claude already made, and attaches its answer.

Classifier

A small, fast model that puts a prompt into one of a few labels.

route asks one (TypeSafe's Jev or ScaleDown's Classify) whether a prompt is a lookup, an explanation, a small edit, a rename, a feature, tests or other. Only feature or test work with a score of at least 0.7 gets the order to hand off.

95% interval

The range the true difference most likely lies in, given the runs.

Differences are reported for the average task with 95% bootstrap intervals, resampling runs within each task. An interval that crosses zero means not shown, and the lab says so.

Graded run

One benchmark run whose result is checked by a fixed rule.

Hidden tests for code, a parser for YAML, a term list for explanations. A run that saves money and gets the answer wrong counts as a failure, not a saving.

List prices

How cost is counted: API prices, request by request from the transcripts.

Opus 5.5 at $4 per million input tokens, $20 output, $0.20 for a cache read, $8 for a cache write. On a subscription these are API-equivalent estimates, not a bill.

Ablation

Taking one mod out at a time to see what is lost.

It showed which mod earns the pack's savings: cells, not the one expected.

Preregistration

Writing down what would count as confirmed before running again.

The first good number is usually too good. Before a confirmation run the lab writes down the one measure and the rule, then runs fresh repeats.