Erased tokens in a grid whose trace lingers in downstream tokens

Your KV Cache Remembers the Context You Deleted

In The Evolution of Agent Memory I argued that once an LLM becomes the brain of an agent, context management becomes unavoidable: interactions pile up, the window eventually fills, and something has to go. Today’s mainstream answers are two — truncate the oldest text and re-prefill what’s kept, or selectively evict tokens by importance. This post is about a third path, and it’s training-free: take the KV cache you prefilled over the full context, slice it down to a subset, and use that — it is often more accurate than recomputing from the same kept text. Note that the two arms keep the exact same text; the only difference is where the KV came from. ...

June 14, 2026 · 8 min · 1685 words · Oklens
Token-level structures of agent memory: flat, graph, tree, hierarchy

The Evolution of Agent Memory

As the LLM completes its transition from a language model into the brain of an agent, it increasingly needs to manage context in a persistent, long-lived way. This poses new challenges for the established RAG and memory techniques. This post looks at the problem from that angle and surveys the most recent research progress in memory management. 1. Existing Memory Evaluation and Architectures Let’s first ask: what are the shortcomings of the memory evaluations and architectures that the academic community takes for granted? ...

March 15, 2026 · 19 min · 3884 words · Oklens