September 2, 2026
From 34% to 90%: caching an agent’s prompt
One line that changed on every step was costing us two thirds of our cache. The fix was not to cache harder. It was to move the line.
An agent that works on a canvas has to be told what's on the canvas. We put that in the system prompt, which is the obvious place to put it, and it quietly cost us most of the benefit of prompt caching.
The rule, which is not obvious
Model providers cache the beginning of a request so they do not have to re-read it every time. The catch is where they measure from. They reuse the longest stretch of text that is identical to last time, counting from the very first character, and they stop at the first difference. So one line that changes on every step ruins the cache for everything after it. It makes no difference where in the request that line sits. Even a conversation history identical to last time has to be read again.
Canvas state changes on every single step. Putting it near the front of the request meant the very first thing the provider compared was already different, every single time.
What we did
We split the prompt by how often things change instead of by what they're about:
- The stable half holds the core instructions, the surface reference, the project name, the brief and memory. It's constant for a whole run, so it makes a good opening that can be re-used.
- The volatile half holds the plan, the current part, canvas state and the step instruction. It's sent at the very end, after the conversation so far, instead of up front with the instructions.
What it was worth
The model that decides what to do next is the expensive part of any run, so that is what we counted. Across 24 test scenarios, which came to 63 calls to that model:
| Canvas state placement | Input tokens | Cached |
|---|---|---|
| Up front with the instructions | 343,987 | 118,114 (34%) |
| Sent last, after the conversation | 351,437 | 316,874 (90%) |
Slightly more tokens in total, and nearly three times as many of them cached. The extra tokens are the framing that moving the block needed, and the cache pays for them many times over.
The rule we wrote down afterwards: never move canvas state back into the system prompt. Moving it back reads as tidier, and it costs two thirds of the cache.
Why this probably applies to you too
Any agent with a stable brief and a volatile world state has this shape. The instinct is to assemble one coherent prompt that describes everything nicely. Prefix caching punishes exactly that. Order your request by how often each part changes, not by how related the parts are.