DeepSeek's on-disk KV cache: exact prefix matching, 890 bytes per token, and a 50x input price gap
On DeepSeek's price list, a million input tokens cost $0.003 when they hit the disk cache and $0.15 when they miss, off-peak rates on deepseek-flash. The same 50x gap holds at peak rates, and the ratio on deepseek-v4-pro is 30x. Nothing about the model changes between those two numbers. What changes is whether the service has seen the beginning of your request before.
The artifact behind the discount is a prefill result stored on disk: the computed K/V state for a prefix of the input. On a hit, the service loads that state instead of re-running prefill over the same tokens, and decode proceeds exactly as it would have. Since 2024 the feature has been on by default for every account, with no code changes required. What it does demand is that the prefix arrive byte-identical, which turns a pricing story into a serialization story.

What the disk cache stores
Two different caches get called "the cache", and only one of them shows up on a bill. The KV cache is internal to a single request: it holds the K/V tensors computed so far so that each new decoded token does not re-derive them. It saves compute, and it has no price of its own. A prompt or prefix cache works across requests: the service keeps the prefill output for a prefix it has already processed, and a later request that starts with the same tokens skips prefill entirely.
DeepSeek's version of the second one lives on disk rather than in VRAM. The 2024 announcement describes caching expected-to-be-reused content on a distributed disk array, retrieving repeated parts instead of recomputing them, and credits the MLA attention in DeepSeek-V2 for making the context KV cache small enough to store on cheap disks in the first place. Storage for the cache is free, and there is no separate fee to write an entry; hits are billed at the cache-hit rate and everything else at the miss rate.
Two properties of the design are easy to misread. Hits only affect the input prefix, so the output is still generated by ordinary inference and still moves with temperature; a cached request and an uncached one can return different text. And the cache is best-effort by design: hits are not guaranteed, and entries that stop being used are cleared automatically, usually within hours to days.
Prefix units and the append-only rule
The matching rule is stricter than a hash of "similar" prompts. Only requests whose prefixes are identical starting from the 0th token count as duplicates, so a mid-input match does nothing. Since the V4 generation, storage and matching work in cache prefix units. The documentation attributes this to Sliding Window Attention: a prefix is stored as one or more complete units, and a later request has to match a unit fully to hit it.
Units get persisted at three kinds of moment. Each request produces one at the end of the user input and another at the end of the model output. The system also detects a common prefix across requests and persists it as its own unit. For long inputs and outputs, units are carved out at fixed token intervals, so a very long prefix does not stay uncacheable just because it never reaches an end position. The 2024 announcement adds a storage floor: the unit is 64 tokens, and anything shorter is not cached at all.
The worked examples in the guide read as a specification. Round one sends A + B, round two sends A + B + C: round two matches the unit A + B and hits. Round one sends A + B, round two sends A + C: no hit, because A + C is not a full match for A + B. Only at that point does the system notice the shared prefix and persist A as a unit, so round three, A + D, is the first request that can hit. Conversation history therefore has to grow by appending. Reordering messages, rewriting an earlier turn, or letting a serializer emit the same tool definitions with a different key order all break alignment and cost the whole prefix.

KV cache compression across generations
Whether a prefix cache can live on disk is a question about bytes per token. The V4 paper measures its own progress on that axis: at a one-million-token context, DeepSeek-V4-Pro needs 27% of the single-token inference FLOPs and 10% of the KV cache of DeepSeek-V3.2. That comes from a hybrid attention stack. Compressed Sparse Attention first compresses a group of KV entries into one and then runs sparse attention over the compressed entries, which suits local detail. Heavily Compressed Attention consolidates more tokens per entry and keeps the attention dense, which suits global mixing. Both branches keep a small set of uncompressed sliding-window entries alongside the compressed ones, and both use learnable attention-sink logits added to the denominator of the attention score, which lets a head put very little weight on the context.
The V4.1-Flash paper pushes the same budget further and describes the numbers more plainly. Its 40-layer backbone is split into a 20-layer causal encoder and a 20-layer decoder; the first two encoder layers use sliding window attention and the rest use Compressed Sparse Attention 2, which reuses KV and indexer state across layers instead of keeping a copy per layer, with global K/V caches stored in FP4. Together those changes put the global KV cache, which always sits in HBM, at 890 bytes per token, about a quarter of what DeepSeek-V4-Flash needs. A deployment optimization called SWA Bounded Replay takes the persistent footprint, the part that lives on SSD or in host memory, to roughly an eighth of the previous generation. Against DeepSeek-V1, the paper puts the per-token global KV cache reduction at 437-fold.
The motivation is stated in the V4.1 paper's first paragraph: long-horizon agents have made workloads input-heavy, prefill is still expensive, and large KV caches strain HBM, SSD capacity and transfer bandwidth at once. Agents are the case that pays for this work, and the case that depends on it.

The disk tier's own trade-offs
Writing compressed attention state to disk is the cheap part. The uncompressed sliding-window state is the expensive part, and the V4 paper spends a subsection on three strategies for it. Persisting all of it leaves no computation to redo, and it produces exactly the access pattern SSDs are worst at: only a small subset of the stored window entries is read by any given hitting request, so the writes are heavy and uneven. Periodic checkpointing stores the window entries of the last tokens every N tokens, and pays for it by recomputing the gap when a prefix is restored. The third option keeps no sliding-window state at all, relying on the compressed CSA and HCA caches and recomputing the recent window on load.
Which one wins depends on where the money is. A write-heavy NVMe workload costs differently from a few seconds of redundant prefill on a GPU, and neither the paper nor the API documentation quotes a crossover point. The full on-disk strategy is the one I would avoid on a consumer SSD, where an unbalanced write pattern is the harder failure to notice. What the documentation does say is that cache construction takes seconds and that the whole feature is best-effort.
Prices, ratios, and latency
The published rates, per million tokens, off-peak, with peak in parentheses:
| Cache hit | Cache miss | Output | |
|---|---|---|---|
deepseek-flash |
$0.003 ($0.006) | $0.15 ($0.30) | $0.60 ($1.20) |
deepseek-v4-pro |
$0.022 ($0.044) | $0.66 ($1.32) | $1.98 ($3.96) |
Off-peak is half of peak, and peak covers two windows on weekdays, 01:00-04:00 and 06:00-10:00 UTC. The 2024 launch priced hits at $0.014 per million and claimed a cut of up to 90% against standard input; the current Flash hit rate is a further 4.7x below that launch price. Latency moved the same way: for a 128K prompt with a high repeat rate, the announcement quotes first-token latency falling from 13 seconds to 500 milliseconds.
Two ratios matter more than the headline figure. Against a miss, a hit is 50x cheaper on Flash and 30x on Pro. Against output tokens, a Flash hit is 200x cheaper, and that is the ratio an agent actually lives on, because every agent turn re-reads a long context and writes a short one. Anthropic's prompt caching, which lives in memory rather than on disk, prices a cache read at a tenth of the input rate and charges extra to write the entry, with a choice of 5-minute and 1-hour lifetimes; DeepSeek charges nothing to write and does not ask you to declare a lifetime, which is the trade that comes with an hours-to-days, best-effort cache.

Hit rates in real harnesses
Everything above is the vendor's side of the deal. The engineering side shows up in third-party measurements, and those are less tidy than the 99% figures that circulate. An engineering write-up from Omni Analytics, benchmarking three models inside one agent harness, reports a main-loop cache hit rate of 94% for Sonnet 5, 93% for DeepSeek and 81% for GLM, and diagnoses the busts. The largest was a single tool description that embedded the current list of field names, which changed whenever a query ran. Tool descriptions sit at the very top of the prompt, ahead of the system prompt, so changing that list invalidated everything after it. In one evaluation run that tool was present on 525 of 570 calls, invoked zero times, and lined up with 47 of the 48 full cache busts. Other busts came from a search tool rewriting its own earlier results when the conversation's topic changed, which dropped reuse on the next call to 69% while neighbouring calls held at 99%, and from a tool list that shrank when a per-turn query cap was hit. Their summary of the failure mode: nothing errors, the answer looks the same, and you pay several times more for tokens you already sent.
The same harness report has a cost table for the same model served by Fireworks: $0.22 per million uncached input tokens against $0.007 cached, with output at $0.66. Across 200 runs, DeepSeek V4 Flash cost $7.28 against $65.15 for Sonnet 5 and $5.24 for GLM 5.3 Flash, with DeepSeek's median prompt at 395k tokens and median output at 4,180 tokens. Cheap input did not make it the cheapest system, and the report's own limitation is the reason to read those totals carefully: in the worst traces the model looped on a failing query, and a low token price does not help if the loop never breaks. Tokenizers cut the other way too, and the report counts 3.9 characters per token for DeepSeek against 2.7 for Sonnet.
Higher numbers exist. A benchmark by Composio comparing eight agent harnesses on DeepSeek V4 Flash is widely reported as putting a harness called Pi at a 99.93% input-token cache hit rate and the lowest average cost per successful task, at roughly $0.028 against about seven times that for the most expensive harness in the set. Those are the project's and the aggregators' figures rather than a controlled independent run, so I would treat the ordering as the finding and the exact percentages as a property of one harness on one workload. On the other end, an issue filed against DeepSeek's repository reports Flash hit rates of 40-50% where Pro held above 90% under the same conditions; it was closed as stale, and it stands as an unconfirmed user report rather than a defect the vendor acknowledged.
A hit rate is a property of the harness, not of the model. The causes in the Omni report are all serialization and ordering decisions made by software: keep the system prefix static and move timestamps, session ids and user identifiers into the latest user message; append to history instead of rewriting it; store raw tool-result strings from the API and replay them verbatim rather than re-serializing parsed objects; sort tool definitions deterministically. And put prompt_cache_hit_tokens and prompt_cache_miss_tokens on a dashboard, because a cache bust is invisible everywhere else.
References
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression (arXiv:2609.19969)
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence (arXiv:2606.19348)
- Context Caching - DeepSeek API Docs
- Models & Pricing - DeepSeek API Docs
- DeepSeek API introduces Context Caching on Disk, cutting prices by an order of magnitude
- DeepSeek-V4.1-Flash: Smarter, Faster, More Efficient
- Toward an AI harness that carries its own weight - Omni Analytics
- Issue #1465: v4-flash cache hit rate abnormally low - deepseek-ai/DeepSeek-V3
Loading comments…