Retrieval is not memory: what recent research shows about agent memory

Many agent platforms start every session from scratch by default, and better document search alone does not change that. Recent research suggests that writing, updating and forgetting are hard to get right, that raw records plus strong search are a hard baseline to beat, and that recall benchmarks such as LoCoMo say little about whether memory helps an agent do its job. We think memory in production should be treated like any other business record: it has a source, a date, an owner and a way to delete it.

Retrieval is not memory

Retrieval-augmented generation (RAG) searches a document store and pastes the best matches into the prompt. It is good at finding what people already wrote down. It does not record what the agent did, what worked, what failed or what changed since last week.

In our view, that gap limits how far stateless agents, which keep nothing between sessions, can go inside a company. Anthropic's documentation for its managed agents states the default plainly: "When a session ends, any state the agent built up is gone" [1]. An agent can look up the refund policy. It cannot learn that refunds for one large customer always need a manager's sign-off, unless someone writes that down where it searches. The authors of ReasoningBank describe the result: agents that approach each new task in isolation "repeat similar errors observed in the past" and "discard valuable insights gained from related problems" [2].

Bigger context windows, the amount of text a model can read at once, have not closed the gap so far. In a July 2025 report, Chroma, which builds search infrastructure for AI, tested 18 models and found performance grew less reliable as input length grew, even on simple tasks [3]. Models read text as tokens, which are word pieces. On questions taken from the LongMemEval benchmark, every model it tested there did better with a hand-prepared focused prompt of about 300 tokens than with the full chat history of about 113,000 tokens [3]. Newer models may narrow this gap: Anthropic reports 76% for Claude Opus 4.6 against 18.5% for Sonnet 4.5 on a recall test at 1 million tokens, a vendor figure on a narrower task [22].

The BEAM benchmark, with conversations of up to 10 million tokens, found that even models with 1-million-token windows "struggle as dialogues lengthen," with or without retrieval [4].

Plain retrieval has limits of its own. In MemoryAgentBench, the strongest RAG agents beat the base model at finding specific facts, though only 5 of the 12 did so on the average of the retrieval tasks, and the authors suggest RAG is likely a poor fit for tasks that need information integrated across a large, coherent context [5].

LongMemEval-V2, released in May 2026, asks whether memory can help agents become "knowledgeable colleagues in customized environments," using histories of up to 500 past runs and 115 million tokens [6]. The strongest RAG baseline scored 48.5%, and the authors' own RAG method reached 57.8%. Their method that stored past runs as files and had a coding agent gather evidence from them reached 72.5%, at a latency of roughly 110 to 140 seconds per query, and an off-the-shelf coding agent baseline scored 69.3% [6]. The paper is a preprint its authors label work in progress.

Memory is the agent's own record, kept current. It helps to split it into three kinds, alongside the working memory of the current context window:

Knowing the company itself, such as who owns what and which system holds the truth, is the subject of our companion article on organizational context.

How memory is written, updated and forgotten

Many memory systems use a language model to decide what to store. Mem0's original design, described in its April 2025 paper, is a clear example: it pulled candidate facts out of a conversation, and the model then added a new memory, updated an existing one, deleted one that new information contradicted, or did nothing [7]. Mem0 has since said that overwrites and deletes sometimes lost details that mattered, and in April 2026 it moved to an add-only extraction step that stores a new fact alongside the old one [21].

Zep takes a time-aware approach. Its Graphiti engine stores facts in a knowledge graph, a network of entities and the links between them [8]. Each fact carries two timelines: when it held true in the world, and when the system learned it [8]. When a new fact contradicts an old one, Zep does not delete the old fact. It marks it invalid from the moment the new fact became valid and gives priority to the newer information [8]. The agent can then answer both "who owns this account now?" and "who owned it in March?"

Updating is one of the weak spots. MemoryAgentBench tests selective forgetting: noticing that a fact has changed and reasoning with the new value. On the multi-hop version, where the answer depends on several linked facts, the authors report that all methods fail: a long-context GPT-5-mini reached 28%, and every RAG or memory agent stayed at 7% or below, including Zep at 3% [5]. The original LongMemEval, on 2024 systems, found overall accuracy drops of 30% or more for commercial chat assistants and long-context models over sustained interactions; that figure is not specific to updates [9].

Consolidation, where a model rewrites many past episodes into short lessons, can make memory worse. A May 2026 preprint of several published methods found that "memory utility first rises, then degrades, and can fall below the no-memory baseline" [10]. Even when consolidating from correct solutions, GPT-5.4's accuracy fell to about 54% on a 19-problem set of ARC-AGI puzzles it had previously solved without memory, so it failed on roughly 46% of them (the paper's abstract says it failed on 54%, but its body reports 46%) [10].

In a controlled test where agents could choose, they kept raw episodes and doubled the accuracy of agents forced to consolidate. The authors advise treating raw episodes as "first-class evidence" and gating consolidation instead of running it after every interaction [10].

Anthropic's agent platform documentation, as of November 2026, describes the problem that pushes teams toward consolidation: incremental writes leave a store cluttered with repeated, conflicting and outdated entries [11]. Its answer, called dreaming and still a research preview, is itself a consolidation step, but a reviewable one: it reads the store plus up to 100 past sessions and writes a new, reorganized store. The input store is never modified, so a person can review the result and discard it [11].

The pattern to copy: keep the raw record, derive facts and lessons from it, version every change, and make consolidation a step you can review and roll back.

Managing context inside one long task

Long tasks raise a different problem. The context window fills with tool output and dead ends. Researchers call the result "context rot": as the number of tokens grows, the model's ability to recall information from its context drops [3][12]. Anthropic's engineers advise treating context "as a finite resource with diminishing marginal returns" [12]. Chroma adds that whether information is in the context is not all that matters: "what matters more is how that information is presented" [3].

Anthropic describes three techniques for long tasks [12]. Compaction summarizes a conversation near the window limit and restarts from the summary, keeping decisions, open bugs and implementation details while dropping redundant tool output. Structured note-taking has the agent write notes outside the window and read them back later. In one example, Claude playing Pokémon kept precise tallies and maps across thousands of game steps. Anthropic has released a file-based memory tool for this pattern [12].

Sub-agents explore in their own clean windows, sometimes using tens of thousands of tokens, and hand back a summary of 1,000 to 2,000 tokens to the main agent [12].

Compaction is lossy, and what it loses can matter. A June 2026 preprint measured what happens to rules an agent has been told to follow [13]. Across 1,323 episodes and seven model families, violations rose "from 0% with the policy in full context to 30% after compaction, reaching 59% for some models." When the rule survived the summary, violations stayed at zero [13]. The study covers rules delivered in the conversation, memory or tool output, on nine simulated tasks; a rule kept in a preserved system message did not decay [13].

The author also showed that hostile content in the context can push the summarizer to drop a policy. Keeping rules out of the text that gets summarized, which the paper calls constraint pinning, brought violations back to 0% in its benchmark, although a fake operator notice in recent context still pushed them to 17% [13].

One preprint is not settled evidence, but the precaution is cheap: keep policies and the task goal in a preserved system message, a pinned section or a durable note, where a summarizer cannot rewrite them.

Memory as a path to self-improvement

Some of the most useful recent work uses procedural memory to let agents improve without retraining. The agent keeps a playbook of strategies, edits it after each task and reads it before the next one. Because the playbook is plain text, people can read and correct it.

ACE, presented at ICLR 2026, adapts the context instead of the model's weights and treats it as "evolving playbooks" built through generation, reflection and curation [14]. Its authors name two failure modes of naive rewriting: "brevity bias," which drops useful detail in favor of short summaries, and "context collapse," where repeated rewriting erodes detail over time. Its authors say ACE avoids both with small, structured updates.

ACE reports gains of 10.6% on agent tasks and 8.6% on finance tasks, and it learned from execution feedback without labeled data, though its authors note it can degrade when that feedback is unreliable. On the AppWorld agent leaderboard, as of September 2025, it scored 59.4% against 60.3% for the top production agent while using a smaller open-source model [14].

ReasoningBank, also at ICLR 2026, distills strategies from both the successes and the failures of past tasks, as judged by the agent itself [2]. Its authors report up to 20% relative improvement and up to 16% fewer steps on web browsing and software engineering benchmarks; on WebArena the gain is 4.6 to 8.3 percentage points over a memory-free agent [2]. Earlier systems mostly learned from successes, and the authors argue that failures are an underused source of lessons [2]. In the later MemoryArena benchmark, ReasoningBank averaged 0.12 success, below the RAG baselines [16].

Skills are the packaged form of the same idea: reusable task know-how that an agent finds and loads when needed. Earlier skill benchmarks mostly handed the agent a hand-written skill for each task [15]. An April 2026 preprint instead tested a ladder of settings, ending with a pool of about 34,000 real-world skills and no hand-picked ones. It found that "the benefits of skills are fragile": gains shrank toward the no-skill baseline as conditions became more realistic [15].

Refining skills for the specific query recovered much of the loss when the starting skills were reasonably relevant. On Terminal-Bench 2.0, a test of command-line tasks, skill retrieval plus refinement lifted Claude Opus 4.6 from 57.7% to 65.5% [15].

Keep the May 2026 consolidation preprint in mind here. ACE was among the methods it tested, and an episodic-only control that simply kept the raw trajectories "remains competitive with the consolidators we test" [10]. We would add a gate of our own: a new lesson goes live only if it beats the current playbook on tasks it was not learned from.

What the benchmarks actually show

Memory benchmarks have moved quickly from recall quizzes toward real work:

Recall is not use. MemoryArena found that agents with "near-saturated performance" on LoCoMo "perform poorly" when memory has to guide later actions [16]. Adding an external memory system or RAG to GPT-5-mini did not consistently beat simply keeping the full history in context, and dedicated memory systems did no better than plain retrieval: across its environments, average success was 0.12 to 0.14 for Letta, Mem0, Mirix and ReasoningBank against 0.17 to 0.20 for the RAG baselines [16].

Full context is a strong baseline while histories are short. There, memory systems have mainly won on cost and speed, as Mem0's April 2025 paper shows [7]. The case for memory looks stronger once history outgrows a window, but the evidence is mixed: BEAM's memory system gained more over long-context models as conversations grew, yet plain RAG beat it on one model at 10 million tokens [4], and the best LongMemEval-V2 result came from searching raw past runs [6].

Scores are also noisy. Penfield Labs, which builds memory systems, published an audit of LoCoMo that has not been peer reviewed. It reports that 99 of the 1,540 questions normally scored (6.4%) have wrong answer keys, which puts the ceiling for a perfect system near 93.6% [17]. Using the judge setup from published evaluations, the LLM judge (a model that grades the answers) accepted 62.81% of answers that were deliberately vague but on topic, against about 11% of answers that were specific but wrong [17]. Treat vendor scores on these tests as claims until you reproduce them on your own data.

Risks: poisoned memories, privacy, deletion and access

Memory turns a one-time attack into a lasting one, because what is written today is read back in later sessions [18]. In the MINJA attack, someone with only ordinary query access gets the agent to write malicious records into its own memory. The authors report an average injection success rate of 98.2% and an average attack success rate of 76.8% in their tests [19].

A January 2026 preprint on health-record agents found that "realistic conditions with pre-existing legitimate memories dramatically reduce attack effectiveness" [20]. It also found that defenses need careful tuning, so that they neither block good entries nor miss subtle attacks [20].

Anthropic's documentation names the general risk. A prompt injection is an instruction hidden in content the agent reads. If an agent with write access reads untrusted input, an injection can plant content in memory, and "later sessions then read that content as trusted memory" [1]. It recommends read-only access for reference material, shared lookups and any store the agent does not need to change [1].

Access control has to travel with the memory. Anthropic suggests one store per end user, per team or per project, and separate stores "when different parts of memory have different owners or access rules" [1]. A fact learned in one customer's session must never surface in another customer's.

Deletion is harder than it looks. In Anthropic's design, as of November 2026, every change creates an immutable version, and versions "are not deleted when the memory itself is deleted." They are retained for 30 days after they are written, although recent versions of a live memory are kept regardless of age [1]. A separate redact step exists for "user deletion requests" [1].

Derived memories raise the same issue in a less visible way. A personal detail can survive inside a summary, a playbook entry or an embedding (the numeric index used for search) after the source record is gone. A 2026 security survey, accepted to EMNLP 2026, argues that provenance, versioning and policy-aware retention should be combined with retrieval and execution defenses, and warns that deleting a visible entry can leave residues in summaries and retrieval indices [18].

Wrong memories are the quietest risk. A stale fact or a bad lesson looks the same as a good one when it is read back. That is why every memory needs a traceable source and a history [10][18].

What to do next

Pick one workflow that repeats across sessions, such as support escalations or month-end close. Then:

1. List what the agent should remember (episodes, facts, procedures) and who owns each kind.

2. Keep raw traces of every run as the source of truth. Derive facts and playbooks from them, with a link back to the source.

3. Put a date range and a source on every fact. Mark old facts invalid instead of overwriting them.

4. Run consolidation as a reviewed batch job with a diff, a test and a rollback.

5. Scope memory by tenant, user, team or project. Default to read-only. Treat anything written from untrusted input as untrusted.

6. Map deletion. For one record, find every place it can live, including versions, summaries and embeddings, then test a full purge.

7. Pin policies and the task goal outside anything that gets compacted.

8. Build an evaluation set from your own traces before you buy or build.

Then measure, on your own data:

• Task success across sessions with memory, without memory and with the full history in context.

• How often the agent repeats a mistake it has made before.

• How often it uses a fact that has been superseded.

• How often it correctly says it does not know.

• Tokens and p95 latency per task.

• Rule violations after compaction.

• Injection success in red-team tests.

• Deletion completeness and time to purge.

• Memory growth, and the share of memories that are never read.

How we think about this at OctaNeural

We treat agent memory as company data that needs owners, history, access rules and tests, rather than as a cache inside one model. We help companies solve problems like this through custom deployments and customizations, so weigh our view with that in mind. That work includes self-improving agents and small custom models trained on a client's own data and agent traces, deployed on-prem where needed, with data and ownership terms set by each agreement. We are also building Memspire, an open-source learning layer for agents (launching soon) that turns every graded run into a lesson and briefs the next run with what it learned. If you are working on agent memory in production, bring it to us.

Sources

1. Anthropic. "Using agent memory." Claude Platform Docs (Managed Agents, beta), accessed October 2026. https://platform.claude.com/docs/en/managed-agents/memory

2. Siru Ouyang, Jun Yan, I-Hung Hsu, et al. "ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory." ICLR 2026 (arXiv v2), March 2026. https://arxiv.org/abs/2509.25140

3. Kelly Hong, Anton Troynikov, Jeff Huber. "Context Rot: How Increasing Input Tokens Impacts LLM Performance." Chroma Research, July 2025. https://www.trychroma.com/research/context-rot

4. Mohammad Tavakoli, Alireza Salemi, Carrie Ye, Mohamed Abdalla, Hamed Zamani, J Ross Mitchell. "Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs." ICLR 2026 (arXiv v2), February 2026. https://arxiv.org/abs/2510.27246

5. Yuanzhe Hu, Yu Wang, Julian McAuley. "Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions." ICLR 2026 (arXiv v4), June 2026. https://arxiv.org/abs/2507.05257

6. Di Wu, Zixiang Ji, Asmi Kawatkar, Bryan Kwan, Jia-Chen Gu, Nanyun Peng, Kai-Wei Chang. "LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues." arXiv, May 2026. https://arxiv.org/abs/2605.12493

7. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, Deshraj Yadav. "Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory." arXiv, April 2025. https://arxiv.org/abs/2504.19413

8. Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, Daniel Chalef. "Zep: A Temporal Knowledge Graph Architecture for Agent Memory." arXiv, January 2025. https://arxiv.org/abs/2501.13956

9. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, Dong Yu. "LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory." ICLR 2025 (arXiv v2), March 2025. https://arxiv.org/abs/2410.10813

10. Dylan Zhang, Yanshan Lin, Zhengkun Wu, Yihang Sun, Bingxuan Li, Dianqi Li, Hao Peng. "Useful Memories Become Faulty When Continuously Updated by LLMs." arXiv, May 2026 (revised August 2026). https://arxiv.org/abs/2605.12978

11. Anthropic. "Dreams." Claude Platform Docs (Managed Agents, research preview), accessed October 2026. https://platform.claude.com/docs/en/managed-agents/dreams

12. Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, Jeremy Hadfield (Anthropic Applied AI). "Effective context engineering for AI agents." Anthropic Engineering, September 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

13. Shiyang Chen. "Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents." arXiv, June 2026. https://arxiv.org/abs/2606.22528

14. Qizheng Zhang, Changran Hu, Shubhangi Upasani, et al. "Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models." ICLR 2026 (arXiv v3), March 2026. https://arxiv.org/abs/2510.04618

15. Yujian Liu, Jiabao Ji, Li An, Tommi Jaakkola, Yang Zhang, Shiyu Chang. "How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings." arXiv, April 2026. https://arxiv.org/abs/2604.04323

16. Zexue He, Yu Wang, Churan Zhi, et al. "MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks." ICML 2026 (arXiv v2), September 2026. https://arxiv.org/abs/2602.16313

17. Penfield Labs. "We audited LoCoMo: 6.4% of the answer key is wrong and the judge accepts up to 63% of intentionally" (title cut off as published). Penfield Labs on Substack, April 2026. https://penfieldlabs.substack.com/p/we-audited-locomo-64-of-the-answer

18. Zehao Lin, Xixuan Hao, Renyu Fu, Shaobo Cui, Kai Chen, Chunyu Li, Zhiyu Li, Feiyu Xiong. "A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle." arXiv, April 2026 (revised October 2026; accepted to EMNLP 2026). https://arxiv.org/abs/2604.16548

19. Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, Zhen Xiang. "Memory Injection Attacks on LLM Agents via Query-Only Interaction." NeurIPS 2025 (arXiv v5), February 2026. https://arxiv.org/abs/2503.03704

20. Balachandra Devarangadi Sunil, Isheeta Sinha, Piyush Maheshwari, Shantanu Todmal, Shreyan Mallik, Shuchi Mishra. "Memory Poisoning Attack and Defense on Memory Based LLM-Agents." arXiv, January 2026. https://arxiv.org/abs/2601.05504

21. Mem0. "Introducing The Token-Efficient Memory Algorithm." Mem0 blog, April 2026 (updated September 2026). https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm

22. Anthropic. "Introducing Claude Opus 4.6." Anthropic News, February 2026. https://www.anthropic.com/news/claude-opus-4-6

Sources checked November 1, 2026.