26 Memory Architectures
Learning objectives. After this chapter you can:
- distinguish working, episodic and semantic memory, and pick storage for each;
- define a write policy — what gets remembered, when, and who decides;
- handle staleness and contradiction instead of accumulating stale facts;
- test a memory at the write, at the read, and across many sessions;
- reason about consent and deletion in a system that remembers people.
26.1 Three kinds of memory, three different problems
“Memory” in an agent covers three things that share a word and almost nothing else. Separating them is most of the design work, because each has a different lifetime, a different storage answer, and a different failure mode.
| what it holds | lifetime | where it lives | how it fails | |
|---|---|---|---|---|
| Working | the current transcript — the task, recent steps, live tool results | one run | the context window | it fills up. Chapter 25 is entirely about this |
| Episodic | what happened, and when: past conversations, actions taken, outcomes | across runs | append-only log, retrievable | it accumulates. Volume grows without bound and old episodes crowd out useful ones |
| Semantic | durable facts: preferences, entities, relationships, learned rules | until contradicted or expired | a store you can update and delete from | it goes stale, and stale facts are worse than absent ones |
The distinction that matters most in practice is the last column. Working memory usually fails loudly — you hit the window and something raises. Not always: Chapter 25’s whole subject is the quieter version, where everything fits and the middle of a long context stops being used well. Episodic memory fails slowly — the system gets more expensive and less relevant.
Semantic memory fails silently: a fact recorded in March is still returned confidently in July, and nothing in the system knows it has expired. That asymmetry is why most of this chapter is about semantic memory even though it is the smallest of the three by volume.
The human analogy is where the vocabulary comes from and it is worth not leaning on too hard. What matters here is engineering: episodic memory is an event log, semantic memory is a mutable key-value store with provenance, and the fact that people’s memory blurs the two is not a reason for your system to.
You will also meet a fourth kind in the literature: procedural memory, knowledge of how to do the task. The widely used agent taxonomy (CoALA, Sumers et al.) lists working, episodic, semantic and procedural memory. In most systems procedural memory is not a store at all: it is the system prompt, the tool definitions and the code, and it changes when you deploy. It becomes a memory problem only when the agent may rewrite its own instructions — a file of “lessons learned” it edits and reads back at every start. Treat that as the most sensitive write path in this chapter, because a bad entry there changes every later run, not one answer.
What we will look at. First where each kind of memory should live, where the popular answer — embed everything — turns out to be the wrong default for exactly one of the three. Then what gets written and who decides, which is a policy you have already chosen by not choosing it. Then a measurement of what happens when the store holds two facts that disagree, which is the question this chapter can answer cheaply and the one most designs assume away. Then forgetting, which has to be built at the start. Then three practical questions: whether memory is a tool the agent calls or infrastructure it never sees, what survives a crash, and how you test a memory at all. And finally what a system owes the people it remembers, all of which turns out to be schema decisions rather than policy documents.
26.2 Storage: stop reaching for a vector database first
Three kinds of memory means three storage questions, and only one of them is hard. Working memory lives in the context window because that is what a context window is. Episodic memory is a log, and a log is a solved problem. The interesting one is semantic memory, where the default instinct is to embed everything and retrieve by similarity.
It is the wrong default, and the reason is the same one Chapter 25 gave for structured data: similarity search cannot express the operations memory actually needs.
The operations are: get this user’s current preference, update it, delete everything about this person, list what changed since March. Those are lookups and mutations on keys. None of them is a nearest-neighbour query.
| store | what it is genuinely good for | where it fails as memory |
|---|---|---|
| Relational | preferences, entities, anything with an identity you can key on; updates and deletes that actually delete | needs a schema, so it cannot absorb whatever the model decides to remember |
| Vector | episodic recall — have we discussed something like this before | no natural update or delete of a fact; two contradicting memories both retrieve happily, ranked by similarity rather than truth |
| Graph | relationships between entities, multi-hop questions about them | operationally heavy, and most projects never ask a question that needs it |
| Key–value / document | a small structured profile per user, read whole | scales badly once you want to query across users |
The architecture that works for most systems is unglamorous and hybrid:
semantic memory → a table. user_id, key, value, source, written_at
episodic memory → a log, embedded for retrieval, with pointers back
to the full recordFacts go in rows; episodes go in vectors.
A common design mistake in this area is putting facts in vectors without giving them an identity. Not because the storage layer refuses: pgvector rows take ordinary SQL UPDATE, DELETE and upsert (update the row if the key exists, insert it otherwise), and every managed vector store has an id-keyed delete. The problem is an append-only collection with no stable key for the fact.
If nothing enforces one row per (user_id, key) — say, one preferred_channel per user — a corrected preference becomes a second embedding sitting next to the first. Both are retrievable, and similarity search has no reason to prefer the new one. Both update and delete turn out to be requirements rather than nice-to-haves — one for correctness and one for law — and what makes them work is fact identity enforced in the schema. A unique constraint on (user_id, key) works just as well on a pgvector table that also stores the embedding. The point is the key, not where the vector lives; any similarity index is a derived view over it.
26.3 Write policies: the decision nobody makes on purpose
The previous section was about where a fact goes once you have decided to keep it. This one is about that decision, which is the one nobody writes down.
What gets remembered is a policy, and if you do not choose one you have chosen “whatever the model decides”, which is a policy with known failure modes.
| who decides | how it behaves | when to use it |
|---|---|---|
| Rule-based | you write the extractor: these fields, from these events | predictable and auditable; the right default for anything that matters |
| Model-decided | the agent calls a remember tool when it judges something durable |
flexible, and it will remember things you would not have — including things it should not |
| User-confirmed | the system proposes, the person approves | highest precision and the only one that gives real consent; costs an interaction each time |
There is a second axis the table does not have, and it decides your security posture rather than your precision: what can reach this write path. Rule-based writes only what you named. User-confirmed writes only what a person approved. Model-decided writes anything the model read — including tool output and any document it was handed.
Four rules that hold up in production:
Remember decisions and preferences, not conversation. The user prefers invoices at the Levent office is a fact with a shelf life. The paragraph in which they said it is an episode, and belongs in the log rather than the profile.
Record provenance with every write. When it was written, what event caused it, and how confident the extractor was. Without provenance you cannot resolve conflicts, cannot expire anything, and cannot answer why does the system think this about me — which is a question you will be asked.
Never write on a single inferred mention. A preference you inferred once and a preference stated three times are different things. The cheapest useful policy separates the source: write an explicit durable statement the first time the user makes it — they told you — and require a second occurrence or a confirmation for anything you inferred from behaviour.
Never write attacker-controlled text as a fact. This is the rule the other three do not imply, and it is the one that makes memory qualitatively worse than a bad prompt rather than the same problem again. A remember tool the model can call after reading a web page, a document, or a support ticket is a write path an attacker reaches without an account on your system: they put the sentence in the content, the model judges it durable, and it survives every later session. A one-shot injection becomes a permanent one, which is the sentence Chapter 36 picks this chapter up on. The mitigation is the provenance field you already have — record the source, refuse writes whose source is retrieved content rather than a person’s statement, and never let a recalled memory carry instructions into a system prompt.
The failure mode of remembering everything is not storage cost — storage is cheap. It is that the retrieval gets worse: every irrelevant memory is another candidate competing with the relevant one, and Chapter 19’s measurements apply directly. A memory system that remembers everything is a retrieval system with a deliberately degraded index.
26.4 Contradiction, measured
A store that accumulates facts will eventually hold two that disagree. The design question is whether you have to handle that, or whether the model sorts it out when it reads both.
It is a cheap question to answer, so here is the answer. Thirty invented contradictions — the user preferred email in March and phone in July — under three conditions with identical facts. Scored on whether the answer matches the newer value. Half the items list the newer fact first, so position is separated from recency.
Scoring. Every answer goes in exactly one of three buckets, so the columns sum to 100% of the thirty items: newer if it names the July value and only that, older if it names the March value and only that, hedged if it names both, neither, or refuses. The 0.0% in the bare row is 0 of 30, not a rounded small number. Read that row as an information control rather than a test of temporal reasoning: with no dates and no rule, nothing in the context says which fact is newer, so hedging — 83.3% here — is the defensible answer, and a careful model picks neither value. Choosing the newer one 0 of 30 times fits that; choosing the older one 5 of 30 times does not (see below).
qwen3.5:9b, 30 contradictions per condition, n = 30
paired: the same thirty items run through all three conditions
condition chose newer 95% CI hedged newer listed first / last
bare 0/30 0.0% [ 0.0, 11.4] 83.3% 0.0% / 0.0%
dated 10/30 33.3% [19.2, 51.2] 66.7% 33.3% / 33.3%
dated+rule 25/30 83.3% [66.4, 92.7] 16.7% 66.7% / 100.0%Three things, in order of how much they should change what you build.
Timestamps alone are not enough. Adding [recorded 12 March 2025] and [recorded 28 July 2025] moved the model from never choosing the newer fact to choosing it a third of the time. Two-thirds of the time it still declined to choose. If your memory design is “store the date and let the model work it out”, this is the measurement that says it will not.
An explicit rule does most of the work, and not all of it. Adding one sentence — when facts conflict, use the most recently recorded one — took it to 83.3%. That is a large, cheap improvement. Still, one answer in six hedges instead of naming the newer value, though none picks the older one.
The three intervals in the table do not overlap, but comparing intervals is not the right test here either: the three conditions ran on the same thirty items, so the comparison between them is paired. Nothing was correct in bare, so the bare-to-dated step has ten discordant pairs all in one direction: McNemar’s exact test gives p = 0.002. The dated-to-rule step adds fifteen net, and across every split of the discordant pairs consistent with those margins the exact p runs from 6 × 10⁻⁵ to 0.004. The Wilson intervals in the table size each proportion; they are not the test of the difference.
The rule interacts with position, and that is the least comfortable number here. In the dated+rule condition the model was right on 15 of 15 items where the newer fact was listed last, and on 10 of 15 where it was listed first — 100% against 66.7%, two-sided Fisher exact p = 0.042. That is fifteen items per cell and a split chosen after looking, so treat it as a warning worth checking on your own model rather than a law. In bare and dated the split is flat (0.0/0.0 and 33.3/33.3), so this is not a general position effect; it appears only once the rule is there. Read conservatively: with the rule, accuracy is somewhere between two thirds and 100%, depending on an ordering you did not think you were choosing. Which is one more argument for the conclusion below: resolve in retrieval, inject one fact, and there is no ordering left to get wrong.
Conflict resolution belongs in your retrieval code, not in your prompt. Resolve before you inject: return one fact, the one your policy chose, and the model has nothing to be inconsistent about. Published systems moved the same decision out of the answering prompt, in two different ways. Rasmussen et al. (2025) builds the store on a bi-temporal graph — each fact carries both when it was true and when the system learned it — and, when a new fact contradicts an old one, invalidates the superseded edge by recording when it stopped being true, rather than keeping two live facts side by side. Chhikara et al. (2025) resolves conflicts at write time: for each new fact, an LLM call compares it with similar stored memories and chooses to add, update, delete or do nothing. A model still decides there, but once, on one pair, when the fact is written — not every time a question is answered. This measurement is a small local argument for an architecture other people arrived at from the other direction.
The default behaviour is hedging, not lying. In the bare condition the model picked the older fact 16.7% of the time and gave both or neither 83.3% of the time. That is better than confidently picking wrong — but a system that answers “your preference is email, or possibly phone” has failed the user just as thoroughly, and it fails in a form that is harder to detect than a wrong answer. In the bare condition the hedge is the model’s most honest reply; the failure belongs to the system, which gave it no way to decide.
The grey band is the failure people do not plan for. It is not a wrong answer; it is no answer, and it shrinks rather than disappears as you add machinery to the prompt. Note also that the older fact is chosen only in the bare condition (5 of 30), and never once dates are added. Without dates the model cannot know which fact is older; that all five picks landed on the older value may be chance (two-sided exact binomial p ≈ 0.06) or an artefact of how the items were built, and this probe cannot tell which.
Small measurement, narrow claim: one model, thirty items, intervals up to 32 points wide. What it establishes is the ordering — bare < dated < dated+rule — and the practical conclusion, which does not depend on the exact figures: do not leave contradiction resolution to the model when a WHERE clause and an ORDER BY … LIMIT 1 over your precedence columns will do it exactly.
So what should the retrieval side do?
Resolve, then inject. Concretely, four rules that cover almost every case:
- One fact per key. If the store can hold two values for preferred channel, the ambiguity reaches the model. Make the write path an upsert and the ambiguity never exists. An upsert overwrites, though, and list what changed since March needs the old value: copy it to a history table (or to the episode log) before overwriting, or keep every version as a row with
valid_fromandvalid_toand read only the open one — the same idea as the bi-temporal store above. The upsert is also the moment to check precedence: if the new value comes from a weaker source than the stored one, keep the stored one and log the conflict. - Recency wins by default, because it is usually right and it is the rule users expect. Recency of what, though: record the time the fact was true and the time you learned it separately, because a late-arriving report of an old event is not news. Where recency is wrong — a stated preference should outrank an inferred one regardless of date, and an authoritative source should outrank a guess — encode that, explicitly, as a precedence over source and event time rather than over write time.
- Expire on a schedule, not on faith. Anything time-sensitive gets a TTL (time to live: how long the fact counts as valid) at write time. A shipping address ages slowly (periodic confirmation may do instead of a TTL); a current project ages fast.
- Surface conflicts you cannot resolve. If two sources of equal precedence disagree, the honest move is to ask, not to guess — and it is a much better user experience than either answer.
26.5 Forgetting is a feature
Systems that only accumulate get worse, and the degradation is not obvious because nothing errors. Three mechanisms, all cheap to add at the start and awkward to retrofit:
- TTL at write time. Every fact carries an expiry or an explicit “does not expire”. The decision is easy when you are writing the extractor and impossible six months later.
- Decay for episodes. Rank retrieval by relevance and age. An episode from last week is usually more useful than an identical one from last year, and a pure similarity ranking cannot express that.
- Deletion that deletes. Not a
deletedflag that the retriever forgets to filter on. This is Chapter 38’s requirement and it is a good reason on its own to keep facts in a store whereDELETEmeans what it says.
26.6 A tool the agent calls, or infrastructure it never sees?
Everything so far has been about the store. This section is about the wire between the store and the agent, and there are only two ways to run it — they trade the same thing Chapter 23’s levels of autonomy traded: capability against predictability.
Memory as a tool. The agent gets remember(fact) and recall(query) and decides when to use them. It is visible in the transcript, so you can audit what was written and why, and the agent can choose not to bother. The cost is that it is another decision the model can get wrong — and Chapter 24 measured what a badly-specified tool does: it goes uncalled, silently, in 41.7% of cases.
Memory as infrastructure. Before every run, your code retrieves what is relevant and injects it into the context — typically a clearly delimited block labelled as stored data about the user, not as instructions, which is the write-path rule above applied on the read side. The agent never knows there is a memory system; it simply knows things. This is predictable, testable, and costs no model decisions — and it is the right default for the same reason a fixed pipeline is the right default in Chapter 23.
The combination most production systems land on: read as infrastructure, write as a tool. Recall happens automatically because it should happen every time; writing is an explicit act, because it is the one you want in the log. “Write as a tool” does not mean handing the write path to the model: the tool is an explicit, logged act that goes through the write-policy rules above, refuses writes whose source is retrieved content, and asks the user to confirm where needed. The tool is where the rules are enforced, not a replacement for them.
26.7 Sessions, checkpoints, and resume
A long-running agent will be interrupted — a deploy, a timeout, a crash, a user closing the tab. What survives that is a design decision, and the useful framing is that the transcript is a value you can persist, not a thing that lives in a process.
- Checkpoint after every step, not at the end. The unit is one loop iteration: the messages so far, plus whatever your runtime needs to continue.
- Store the tool results, not just the calls. Replaying a run by re-executing its tools is wrong for anything that writes, and slow for everything else.
- Make resume the same code path as start. A resumed run that takes a different branch is a second implementation you will not test.
Packer et al. (2023) is the reference design for taking this seriously — an agent that manages its own paging, the operating-system term for moving data between a small fast memory and a large slow one: here, between the context window and external storage. It is the paper worth reading if you want the operating-system analogy done properly.
26.8 Evaluating a memory
A memory can fail at three points, and each needs its own check.
- The write. Of the facts the system stored, how many should it have stored (precision)? Of the facts it should have stored, how many did it store (recall)? A labelled set of a few dozen conversations, each with the facts a careful person would keep, is enough to start. Count forbidden writes — injected instructions, data you promised not to keep — separately, because one of those matters more than ten missed preferences.
- The read. Given a key or a question, does recall return the current value, and only that? The contradiction probe above is a test of this kind, and so are steps 3 and 4 of the lab.
- The whole loop, over many sessions. Public benchmarks do this with long multi-session conversations and questions about them: LoCoMo (Maharana et al., 2024) and LongMemEval (Wu et al., 2024), whose question types include knowledge updates — this chapter’s contradiction problem — and abstention, answering “I don’t know” when nothing was stored. Chhikara et al. (2025) reports on LoCoMo and Rasmussen et al. (2025) on LongMemEval. Numbers from different papers are rarely comparable: the answering model, the judge and the prompt all differ.
The general machinery — judges, confidence intervals, paired comparisons — is Chapter 30’s subject; what is specific to memory is that the questions have to be asked after time has passed and facts have changed.
26.9 Privacy, consent, and deletion
A system that remembers people has obligations that a stateless one does not, and the engineering decisions that satisfy them are made in Section 26.2 rather than in a policy document written afterwards.
Deletion has to actually delete. This is the concrete reason facts belong in a store with a real DELETE. The right is written into law: the GDPR’s right to erasure (Article 17) in the EU, and in Türkiye KVKK Article 7, which requires personal data to be deleted, destroyed or anonymised once the reason for processing it is gone, with Article 11 giving the person the right to ask; Section 38.1 covers both regimes. Erasing a whole user is the easy case if every vector carries a user_id. “We embed everything” becomes a problem the first time someone asks you to forget or correct one specific thing: a vector index deletes a vector by id, not a fact. Unless you kept a mapping from each fact to the vectors that encode it, you first have to find them. Searching an index for every trace of one fact to honour one request is the cost of the architecture decision, paid later.
Provenance is what makes an answer possible. Why does this system think I prefer phone calls? is a question with a good answer only if you recorded which event caused the write. It is one column, added at the start.
Inference is not consent. A preference the user stated is different from one your extractor inferred, and users experience the second as surveillance even when it is accurate. Store the distinction, use it in precedence rules — and when a memory drives something visible, be able to show what it was based on. Never let the extractor infer health, religion, political views or anything else the law treats as special-category data (GDPR Article 9, KVKK Article 6): an accurate guess about such a thing is itself sensitive processing. And “consent” in this chapter means the everyday sense — the person saw the memory and agreed. Whether that is also your legal basis under KVKK or the GDPR is a separate question; Section 38.1 explains why consent is rarely the basis that fits.
Retention is a decision, not a default. Everything written gets an expiry or an explicit exemption. Chapter 38 is where this becomes a regime; the part that belongs here is that a TTL column costs nothing at write time and is close to impossible to add convincingly after the fact.
The uncomfortable version of this section: most memory systems are built by adding writes until the demos improve, and the privacy work is done at the end by someone else. Almost every requirement above is a schema decision — provenance, source, expiry, deletability — which means it is cheap on day one and expensive on day two hundred.
26.10 Intuition checks
Answer each one before you open it. One is settled by the measurement this chapter runs; the other five are design decisions it argues for, and most of them come down to a schema choice that costs nothing on day one.
Episodes yes, facts no. Similarity search is right for have we discussed something like this and wrong for what is this user’s current preference. Be precise about why, because “you cannot delete from a vector store” is not it — you can, by id. What a vector index does not give you is a key: nothing stops two contradictory embeddings of the same preference coexisting, and nearest-neighbour search has no notion of which is current. Facts go in rows keyed by (user, key), where an upsert makes the contradiction impossible; episodes go in an index with pointers back to the record. Putting facts in vectors without a key is a common production mistake in this area, and you pay for it the first time a user asks you to correct or forget one specific fact.
Not reliably. Measured here: with both facts present and no timestamps, the model chose the newer one 0% of the time and hedged on 83.3%. Adding timestamps got it to 33.3%. Adding an explicit “prefer the most recent” rule got it to 83.3% — better, and still one answer in six hedging. Resolve in the retrieval layer and inject one fact.
Note which direction the failure goes: hedging, not confident error. That is the better failure, but a system that answers “email, or possibly phone” has still failed. A bare accuracy number does not show whether a miss was a hedge or a wrong pick.
Not a single inference. The cheapest policy that works separates the two cases: an explicit durable statement — always email me, never call — can be written the first time, because the user said it; anything you inferred needs a second occurrence or a confirmation. And record provenance with every write, because without it you cannot resolve conflicts, expire anything, or answer why the system believes something. The failure mode of remembering everything is not storage cost but retrieval quality: every irrelevant memory competes with the relevant one.
Semantic. Working memory fails loudly — you hit the window. Episodic memory fails slowly — retrieval gets worse and costs rise. Semantic memory fails silently: a fact recorded in March is returned confidently in July, with nothing in the system aware it expired. That asymmetry is why TTLs belong at write time rather than on a cleanup backlog.
Only where a wrong write is cheap. Model-decided writes are flexible and will remember things you would not have chosen, including things you are not allowed to keep. Rule-based extraction is predictable and auditable and is the right default for anything that matters; user-confirmed writes cost an interaction and are the only mechanism that gives real consent. Most systems want read-as-infrastructure and write-as-an-explicit-act.
The transcript, checkpointed after every step rather than at the end, with tool results stored and not just the calls — replaying a run by re-executing its tools is wrong for anything that writes. And resume should be the same code path as start, because a separate resume path is a second implementation that will not get tested.
26.11 Interview questions
On sourcing — the same caveat as Chapter 2. These recur in the two credible open sources this book uses, plus the ones this chapter’s own measurement makes answerable.
Testing whether you understand the design
Q: How would you design memory for a conversational agent?
Start by splitting the word: working memory is the transcript and belongs to context engineering; episodic memory is an event log, retrieved by similarity; semantic memory is facts, and belongs in rows keyed by user and attribute with provenance and an expiry. Then say what each store must support — facts need update and delete, episodes need ranking by relevance and age — and note that this is why “embed everything” fails: without a key per fact, there is nothing to update.
Q: Two memories contradict. What happens?
Resolve before injection, in code, and give the model one fact. If you leave it to the model, expect inconsistency: measured on a small probe, timestamps alone produced the newer answer a third of the time, and even with an explicit recency rule one answer in six still hedged. The interesting follow-up is where recency is the wrong rule — a stated preference should outrank an inferred one whatever the dates — which is why precedence is over source as well as time.
Testing whether you have shipped one
Q: A user asks you to delete everything you know about them. What happens?
Walk the actual data path: the fact store, the vector index, the episode log, backups, and any cache. The honest answer names the hard one. Deleting a user is usually fine — every store can delete by id. Deleting a fact is where an embedding-only design hurts: you have to know which vectors encode it, which means you needed the mapping from fact to embedding all along, and derived copies inside summaries are harder still. Say that this is a schema decision made on day one, not a feature added later, and that it is the strongest practical argument for keeping facts in a relational store.
Q: Your agent keeps acting on outdated information. Where do you look?
At write time, not read time. Ask whether anything carried a TTL, whether writes upsert or append, and whether retrieval returns one fact per key or several. Most “stale memory” bugs are really “two values for one key” bugs, and the fix is a unique constraint rather than a smarter prompt.
26.12 The 2026 toolbox
The contradiction probe in this chapter ran on one laptop with the versions below. It is short prompts and thirty items, which is why it fits there at all.
| library | version used | what it is for here |
|---|---|---|
ollama |
0.24.0 | serving qwen3.5:9b for the contradiction probe |
numpy |
2.5.2 | task construction and scoring |
Worth knowing about, not used above:
| library | current version | when you would reach for it |
|---|---|---|
letta |
0.33.1 | self-managed memory paging, the MemGPT lineage |
mem0ai |
2.2.0 | extraction, update and conflict policies as a library (Chhikara et al. 2025) |
graphiti-core |
0.30.2 | the open-source temporal knowledge graph behind Zep, when when matters as much as what (Rasmussen et al. 2025) |
langmem |
0.0.30 | memory primitives for LangGraph agents |
pgvector (Python package) |
0.5.0 | episodes in the database that already holds your facts — usually the right answer; the PostgreSQL extension is versioned separately (0.8.x as of writing) |
26.13 Lab: build a memory that can be wrong safely
Six steps, and the first one has no code in it on purpose — a memory system you cannot describe in a paragraph is one whose write policy you have not chosen.
Step 1 — write the policy before the code
Three things your system must remember, three it must never store, and for each of the three it stores: who decides, and when does it expire.
Step 2 — a table, not an index
user_id, key, value, source, valid_from, written_at, expires_at — with valid_from for when the fact became true and written_at for when you learned it. Upsert on (user_id, key) so two values for one key cannot exist, and copy the old value to a history table before you overwrite it.
Step 3 — plant a contradiction
Write a preference, then write a different one. Query it. If your retrieval returns both, fix the retrieval — this chapter measured what happens when the model is left to resolve it.
Step 4 — measure staleness
Simulate six months passing: on a copy of your store, move every timestamp back six months, then ask ten questions. Shifting dates does not change the answers, so judge each one yourself — would this fact probably still be true after six months (an address, yes; a current project, often not)? Count the answers that are probably wrong now. That number is what a TTL is worth, and the keys behind them are the ones that need one.
Step 5 — delete a user, then look
Run your deletion path, then search the raw store, the index, the logs, and any cache for the person’s data. Report where it survived.
Step 6 — answer “why do you think that?”
For three memories, produce the event that caused each write. If you cannot, add the provenance column now.
What to hand in
The policy from step 1, the staleness count from step 4, the list of places data survived deletion in step 5, and one sentence naming the memory your system should stop writing.
26.14 Exercises
- The write policy. For a support agent: three things it must remember, three it must never store, and for each stored item who decides and when it expires. One page, before any code.
- Plant a contradiction. Put two conflicting facts in your store, then decide the resolution rule and defend it against the case where recency is the wrong answer.
- Age your store. Move every timestamp back six months and re-ask ten questions. For each answer, judge whether the fact behind it would probably still be true six months later; count the ones that would not. That is what you are buying with a TTL.
- Delete and hunt. Run your deletion path for one user, then search every store, index, log and cache. Write down where the data survived.
- Provenance audit. Pick five memories and produce the event that caused each. If you cannot for any of them, that is the finding.
- Facts versus episodes. Take your current memory store and classify every entry as one or the other. If facts are in the vector index, estimate what moving them costs now against what it costs in a year.
- Tool or infrastructure. Implement recall both ways on the same task and compare: how often does the agent forget to call the tool, and what does automatic injection cost you in tokens?
26.15 Where to go deeper
Before reading: take your memory store and classify ten entries as fact or episode. If facts are living in a vector index, you have found your first task.
| resource | what it is |
|---|---|
letta |
the MemGPT design as a working system — the clearest way to see self-managed paging between context and storage |
mem0 |
a memory layer with explicit extraction and update policies; useful to read for the policy vocabulary even if you build your own |
graphiti |
Zep’s open-source temporal knowledge graph: facts with validity intervals, which is the contradiction problem taken seriously |
| Building effective agents — Anthropic | the surrounding argument for why less machinery usually wins |
| Lil’Log on agents | the widely read survey post that maps the human-memory vocabulary (short-term, long-term; episodic, semantic, procedural) onto agent components — note that it maps all long-term memory onto a vector store, the default this chapter argues against |
| CoALA — Sumers et al. | the paper behind the working / episodic / semantic / procedural taxonomy for language agents |
| topic | source |
|---|---|
| an agent that pages its own memory | Packer et al. (2023) |
| extraction, consolidation and update as a library | Chhikara et al. (2025) |
| invalidating a superseded fact instead of storing both | Rasmussen et al. (2025) |
| what a long context does and does not retain | Hsieh et al. (2024); Modarressi et al. (2025) |
| why one retriever’s scores do not transfer across corpora | Thakur et al. (2021) |
| testing memory over many sessions | LoCoMo; LongMemEval |
| the loop this all hangs from | Yao et al. (2023) |
26.16 What to remember
- Three memories, three failure modes. Working memory fails loudly when the window fills; episodic memory fails slowly as volume degrades retrieval; semantic memory fails silently, returning a March fact confidently in July.
- Facts in rows, episodes in vectors. A fact needs update and delete, and a vector index offers them only by vector id, not by fact. Putting facts in an index without a key is a common production mistake, and you pay for it the first time a user asks you to correct or forget one specific fact.
- The model will not resolve contradictions for you. With both facts present and no timestamps it chose the newer one 0% of the time and hedged on 83.3%; timestamps alone took it to 33.3%; an explicit recency rule took it to 83.3% and left one answer in six hedging. Resolve in retrieval, inject one fact.
- Hedging is the default failure. Not a confident wrong answer but “email, or possibly phone” — better than lying, still a failed answer, and a bare accuracy number cannot tell it apart from a wrong pick.
- Write an explicit statement the first time, and an inference only on a second occurrence or a confirmation, and record provenance every time. Without provenance you cannot resolve, expire, or explain.
- Test the write, the read and the whole loop. Write precision and recall, “does recall return the current value”, and questions asked after facts have changed.
- Forgetting is a feature. TTL at write time, decay for episodes, and deletion that actually deletes rather than a flag the retriever might forget.
- Read as infrastructure, write as a tool. Recall should happen every time and needs no model decision; writing is the act you want visible in the transcript.
- Privacy requirements are schema decisions. Provenance, source, expiry and deletability cost nothing on day one and are close to impossible to add convincingly on day two hundred.