DeepSeek V4.1 Flash KV Cache: 890 Bytes per Token Explained

DeepSeek V4.1 Flash stores its global KV cache at 890 bytes per token. That is roughly one-quarter of what V4-Flash used in HBM and one-eighth of its SSD storage footprint. Measured against DeepSeek V1, the company describes it as about a 437-fold reduction^[3][5].

KV cache is the working memory an attention model keeps for every token it has already read. At a million tokens of context, even a few kilobytes per token becomes hundreds of gigabytes, so per-token cache size is what decides whether long-context serving is affordable^[3].

V4.1 Flash attacks that number with four mechanisms stacked together: a causal encoder-decoder projection, Compressed Sparse Attention 2, a hierarchical sparse indexer, and FP4 storage for the main KV. Each is documented on the model card and the vLLM recipe page^[1][2].

• 890 bytes per token for the global KV cache^[1]
• Roughly one-quarter of V4-Flash's HBM footprint^[3]
• Roughly one-eighth of V4-Flash's SSD storage^[3]
• About a 437x reduction versus DeepSeek V1^[3]

V4.1 Flash is a Causal Encoder-Decoder model: 40 Transformer layers split into a 20-layer causal encoder and a 20-layer decoder. The decoder's global KV cache is projected from the final encoder hidden states rather than derived separately at every decoder layer^[1].

That projection is the structural reason the cache can be small. Because every decoder layer reads from one shared encoded representation, there is no need to store a full per-layer KV set for the decoder's global attention^[1][5].

In FLOP terms, DeepSeek says CED roughly halves prefill computation on long sequences, while single-token decode FLOPs rise only about 25% as context grows 256-fold from 4K to 1M tokens^[3].

Compressed Sparse Attention 2 assigns every attention layer one of three static modes: Full, Reindex, or Reuse. The modes determine how main KV and indexer keys are shared across layers, so not every layer pays the full storage cost^[1].

Full-mode layers produce the reference representation. Reindex layers rebuild their indexing from that reference, and Reuse layers simply share it. Because the assignment is static, the serving stack knows the cache layout at load time and can plan memory accordingly^[1][5].

DeepSeek notes that CSA2's sparse selection introduces robustness boundaries it has not fully characterised. It warns that sparse-selection errors could degrade capability in untested edge cases, a caveat the model card states openly^[3].

• Full: produces the reference KV and indexer keys^[1]
• Reindex: rebuilds the index from the reference^[1]
• Reuse: shares the reference representation^[1]

A hierarchical sparse indexer restricts later indexing layers to a candidate pool built by the first Full-mode layer. The pool is 2,048 blocks of 8 tokens, and each query keeps its best 512 candidates^[1].

The hierarchy is what bounds cost independently of context length. Instead of every indexer layer searching the entire sequence, deeper layers search an already-narrowed candidate set, so indexer work grows with the pool rather than with the full 1M context^[1].

Combined with CSA2, this is the part of the design that makes sparse attention practical at a million tokens: the model keeps enough signal to retrieve the right blocks while paying for a bounded subset of them^[1][3].

The main KV cache is stored in FP4 using the E2M1 format, with one E4M3 scale per 16 channels. That micro-block scaling keeps precision usable while cutting the bytes per stored value roughly in half versus an FP8 layout^[1].

E2M1 gives four bits per value, so a block of values plus a shared scale is far smaller than the same data at BF16. The per-16-channel E4M3 scale is the compromise that stops outliers from destroying accuracy^[1].

This is a deliberate precision decision, not a lossless shortcut. DeepSeek pairs it with native quantization-aware training for the model weights, so the cache format and the weight format are designed as a set rather than assembled after the fact^[1].

Sliding-window attention needs its own state, and V4.1 Flash does not persist all of it. SWA Bounded Replay reconstructs missing sliding-window states by replaying only the most recent n_win tokens instead of keeping them on SSD^[1].

Bounded replay is what removes a large slice of the storage cost. When short-lived window state is evicted, the server does not reload it from disk; it simply recomputes the recent window, which is cheap because the window is small^[1][3].

The consequence is a two-tier cache: a long-lived global representation that is expensive to lose, and a cheap, reconstructable window state. DeepSeek's design leans on that asymmetry to keep storage requirements down^[1].

DeepSeek guarantees the global, long-lived KV cache a lifetime of at least 72 hours in its persistent cache. The short-lived SWA KV lives in a distributed memory pool provisioned from 10% of host DRAM^[3].

When the global cache survives but the SWA window state does not, V4.1 Flash replays only the most recent window rather than the full history. That hierarchy is why the persistent cache can be useful without being enormous^[3].

For API users, the 72-hour window is what makes cache-hit pricing meaningful on agent workloads: a session that pauses and resumes within three days can still start from a warm cache^[3][6].

• Global KV persistent lifetime: at least 72 hours^[3]
• SWA KV pool: 10% of host DRAM^[3]
• On SWA eviction: replay the recent window only^[3]

Because cache-hit charges can dominate an agent's bill, shrinking the cache changes real economics. DeepSeek prices cached input at a fraction of uncached input, so a smaller cache per token means more of a long context fits in the cheap tier^[6].

The combined effect of CED, CSA2, the hierarchical indexer, FP4 KV and bounded replay is that V4.1 Flash can hold a 1M-token context at roughly a quarter of the HBM and an eighth of the SSD of its predecessor. For input-heavy agents that repeatedly read a repository or tool definitions, that is the difference between a viable and an unviable unit cost^[3][5].

VentureBeat frames the same point from the billing side: caching economics are central to agent costs, and a model that compresses its cache is effectively cheaper per unit of retained context even before the per-token rate is applied^[5].

• Smaller per-token cache means more context in the cheap tier^[6]
• Input-heavy agents benefit most^[5]
• A 72-hour lifetime enables warm restarts^[3]