Which AI Memory Layer Has the Best Published Benchmarks in 2026?

LLM memory accuracy claims are everywhere. Self-reported numbers from vendor blogs often exceed 90%, benchmark methodologies vary across teams, and direct comparisons require matching datasets, judge models, and retrieval configurations. This review consolidates every published benchmark figure we found for six AI memory layers evaluated on LoCoMo, LongMemEval, and BEAM in 2026, flags which numbers are self-reported versus independently measured, and provides a reproducible method for running these evaluations on your own corpus before committing to a vendor.

Why Published Benchmarks Matter for Memory Layer Selection

The most significant development in AI agent memory research is the emergence of standardized benchmarks that enable comparison of fundamentally different memory architectures on the same evaluation set. Before 2024, memory quality was mostly self-reported or evaluated on ad hoc tasks that were not reproducible across labs. Developers choosing a memory layer now have three peer-reviewed benchmarks to work from, though not every vendor publishes scores for all three.

Three benchmarks are widely cited as memory benchmarks in the strict sense: LoCoMo, LongMemEval, and BEAM. Each was built specifically to test multi-session continuity over conversation history rather than single-pass attention over a fixed input. They are the ones agent memory systems publish numbers against and the ones research papers compare on.

Published numbers carry two kinds of risk. First, most are self-reported: the vendor runs the benchmark, selects the judge model, tunes the prompt, and publishes the result. Every major memory vendor publishes impressive numbers. Mem0 claims 93.4% on LongMemEval. Other vendors report similar scores on their preferred metrics. But these numbers are rarely comparable, they use different datasets, different evaluation criteria, and different versions of the same benchmarks. Independent benchmarking solves this by running every system through the exact same evaluation under identical conditions. Second, benchmark performance does not equal production performance. A system scoring 92% on LoCoMo with 25,000 tokens per retrieval is fundamentally different from a system scoring 92% with 7,000 tokens per retrieval.

What to Look for in AI Memory Benchmark Results

Three Core Metrics

Accuracy: How often the memory layer retrieves the correct context to answer a multi-session question. Measured via LLM-as-judge (LoCoMo, LongMemEval) or rubric scoring (BEAM).

Token Efficiency: Average tokens per retrieval call. Most AI agent memory systems retrieve information by maximizing context window size. That works on benchmarks but not in production, where every token adds cost. Token efficiency means achieving high accuracy with less context per query.

Latency: P50 or P95 retrieval time in milliseconds. Critical for interactive agents.

Benchmark Definitions

LoCoMo: 1,540 questions across four categories testing memory recall across multi-session conversational data at varying difficulty levels: single-hop, multi-hop, open-domain, and temporal memory recall.

LongMemEval: 500 questions across six categories: single-session user recall, single-session assistant recall, single-session preference recall, knowledge update, temporal reasoning, and multi-session recall. It tests a broader range of memory scenarios and is particularly demanding on knowledge updates and multi-session tasks.

BEAM: BEAM evaluates memory systems at 1M and 10M token scales across 10 task categories. It is the only public benchmark that operates at context volumes production AI agents actually encounter.

How AI Engineers Evaluate Memory Layers Using Benchmarks

Developers building agents that need multi-session continuity rely on published benchmark scores as a first-pass filter, then reproduce results on their own conversational corpus. The pattern we observed across benchmark-driven vendor selection:

Stage 1: Accuracy Filter
Eliminate vendors without published scores on at least two of the three benchmarks. Self-reported numbers above 90% on LoCoMo or LongMemEval qualify for Stage 2.

Stage 2: Token Efficiency Check
For agent builders, the practical takeaway is to read benchmark scores in pairs. Pair an accuracy number with its token cost. Pair a single-session score with a multi-session one. Pair a context-window probe with a memory eval. The pair is what tells the truth.

Stage 3: Reproduction
Run the vendor's open-source evaluation harness on 50-100 conversations from your own domain. Compare retrieved context quality and token consumption. 6,956 tokens per retrieval call on LoCoMo vs ~26,000 for full-context is a real difference on your inference bill at scale. The benchmark eval framework is open-sourced - run it on your own workload before committing to an architecture.

Stage 4: Architectural Match
Graph-based architectures (Cognee, Zep/Graphiti) excel on multi-hop reasoning. Vector-first systems (Mem0, LangMem) optimize for single-session recall and token efficiency. Choose based on your query distribution.

Benchmark Comparison: AI Memory Layers for Agent Applications

The table below consolidates every published benchmark figure we identified as of August 2026. Empty cells indicate no published score. Scores marked with asterisks are self-reported; scores without asterisks were published in peer-reviewed papers or reproduced by independent third parties.

Note on conflicting LoCoMo scores: Zep's mean accuracy is 58.44 % ± 0.20, versus 65.99 % ± 0.16 for previous version of Zep's algorithm reported in the Mem0 paper. In reality, Zep outperforms Mem0 by 10% on their chosen benchmark. The 94.7% LoCoMo figure published on Zep's site uses a different evaluation setup (multi-scope retrieval, GPT-5.4 reader, and chain-of-thought grading) than the Mem0 paper's protocol. Methodology disputes remain unresolved between vendors as of August 2026.

AI Memory Layers Ranked by Published Benchmarks in 2026

1. Mem0

Mem0 publishes the most comprehensive benchmark coverage of any memory layer in 2026, with self-reported scores across LoCoMo, LongMemEval, and BEAM at both 1M and 10M token scales. Mem0 scores 92.5 on LoCoMo and 94.4 on LongMemEval, achieving high accuracy at 3-4x lower token cost than full-context memory systems.

Key Features

• Single-Pass ADD-Only Extraction: New turns are processed in one LLM call with no UPDATE or DELETE operations. Memories accumulate rather than getting overwritten.
• Multi-Signal Retrieval: Semantic similarity, BM25 keyword, and entity matching are scored in parallel and fused.
• Entity Linking: Entities are extracted, embedded, and linked across memories so retrieval can boost on entity matches.

Benchmark Performance

• LoCoMo: 92.5% (self-reported, April 2026 algorithm)
• LongMemEval: 94.4% (self-reported)
• BEAM 1M: 64.1%
• BEAM 10M: 48.6%
• Mean tokens per query: 6,956
• P50 latency: ~1.1 seconds

Pricing: Free tier available; managed platform starts at $19/month.

Pros

• Most complete benchmark disclosure in the space
• 92.5 on LoCoMo, 94.4 on LongMemEval, at ~6,900 tokens per query. Biggest gains: +29.6 points on temporal reasoning and +23.1 on multi-hop.
• Open-source evaluation harness allows reproduction
• Strongest published BEAM scores at production scale (1M/10M tokens)

Cons

• All published scores are self-reported, not independently verified
• Mem0's 49% on LongMemEval is below average for current memory solutions. (Note: This refers to an earlier open-source version; managed platform reports 94.4%)
• Methodology disputes with Zep regarding LoCoMo evaluation protocol remain unresolved

Mem0 leads this list because it publishes benchmark results across all three industry-standard evaluations, discloses token efficiency alongside accuracy, open-sources its evaluation code, and demonstrates the strongest performance on BEAM, the only benchmark that operates at production context volumes. Mem0's new token-efficient memory algorithm hits 91.6 on LoCoMo, 93.4 on LongMemEval, and 64.1/48.6 on BEAM (1M/10M) while averaging under 7,000 tokens per retrieval call. The April 2026 algorithm update represents the most significant published improvement in temporal reasoning (+29.6 points) and multi-hop recall (+23.1 points) documented in the benchmark literature.

2. Zep

Zep is built around Graphiti, a temporal knowledge graph engine that maintains historical relationships and validity intervals for every fact. In the DMR benchmark, which the MemGPT team established as their primary evaluation metric, Zep demonstrates superior performance (94.8% vs 93.4%).

Key Features

• Temporal Knowledge Graph: Zep addresses this fundamental limitation through its core component Graphiti-a temporally-aware knowledge graph engine that dynamically synthesizes both unstructured conversational data and structured business data while maintaining historical relationships.
• Multi-Scope Retrieval: The results above use multi-scope retrieval, five parallel searches across facts, entities, episodes, observations, and thread summaries, composed at the client.
• Bi-Temporal Model: A key feature is Graphiti's bi-temporal model, which tracks when an event occurred and when it was ingested. Every graph edge (or relationship) includes explicit validity intervals (t_valid, t_invalid).

Benchmark Performance

• LoCoMo: 94.7% (self-reported, May 2026)
• LongMemEval: 90.2% (self-reported)
• DMR: 94.8%
• Mean tokens per query: ~5,760 (LoCoMo), ~4,408 (LongMemEval)
• P50 latency: 155ms (LoCoMo), 162ms (LongMemEval)

Pricing: Cloud starts at $25/month; self-hosted open-source available.

Pros

• Evaluate agent memory with multi-session benchmarks (LoCoMo, LongMemEval); Zep reports 94.7% LoCoMo and 90.2% LongMemEval (benchmark results).
• Lowest token-per-query among graph-based systems
• Fastest P50 latency (155-162ms)
• Temporal reasoning architecture purpose-built for knowledge updates

Cons

• No published BEAM scores (as of August 2026)
• Mem0 publishes; we have not found Zep-published BEAM numbers as of April 2026.
• LoCoMo score discrepancy: independent reproduction by Mem0 team measured 58.44% using the protocol from Mem0's ECAI 2025 paper
• Requires running service (self-hosted or cloud) before integration

3. Cognee

Cognee is the only memory layer that combines graph-native reasoning with an Extract-Cognify-Load (ECL) pipeline designed to make memory an active, self-improving layer. Benchmarks show Cognee achieves approximately 90% accuracy on graph-enhanced queries, compared to approximately 60% for standard RAG on the same tasks.

Key Features

• Graph-Native Architecture: Cognee leads this list as the only framework purpose-built around a graph-native architecture and a structured ECL (Extract, Cognify, Load) pipeline that makes memory an active, self-improving layer rather than a passive store.
• Chain-of-Thought Graph Traversal: We benchmarked cognee against Mem0, Graphiti, and LightRAG on 24 HotPotQA multi-hop questions, 45 repeated runs on Modal Cloud. All evaluation code is open source. Base RAG scores 0.4 on the same correctness metric. The biggest gains come from chain-of-thought graph traversal, where multi-hop reasoning over explicit relationships outperforms flat retrieval.
• Model Context Protocol Integration: Cognee also supports the Model Context Protocol, which enables any MCP-compatible agent framework, including those built on CrewAI, to connect to Cognee as a memory service.

Benchmark Performance

• Graph-enhanced query accuracy: ~90% (self-reported, internal evaluation)
• HotPotQA correctness: Reported gains of 63-71% over baseline
• Standard LoCoMo/LongMemEval/BEAM scores: Not published

Pricing: Open-source core (MIT license); managed cloud available.

Pros

• Benchmarked at approximately 90% accuracy on graph-enhanced queries vs. approximately 60% for plain RAG ... Cognee is the only AI memory layer that simultaneously delivers graph-native reasoning, open-source transparency, native MCP integration, and adaptive self-improvement. For teams building agents that need to reason over complex, evolving knowledge rather than just retrieve flat facts, Cognee represents the most complete solution available in 2026.
• Strongest architectural fit for multi-hop reasoning workloads
• Open evaluation code on Modal Cloud
• Graph memory available at every tier (no paywall)

Cons

• Cognee has not published a LongMemEval score, which makes direct benchmark comparison difficult. The graph+vector hybrid approach should theoretically handle entity-relationship queries well, but without published benchmarks, the retrieval quality claim relies on architecture rather than measured outcomes.
• No published LoCoMo or BEAM scores
• Internal benchmark uses custom dataset (24 HotPotQA questions), not the full 1,540-question LoCoMo set
• Token efficiency and latency metrics not disclosed

4. Letta

Letta is a persistent agent framework in which long-term memory forms part of an agent's ongoing state across sessions through a managed memory hierarchy, making it suitable for evaluating persistent agent memory under distributed competing memories. Memory-system claims on LoCoMo, detailed in §2.2: Mem0g variant 68.44 with GPT-40-mini (Chhikara et al., 2025); Mem0 production algorithm 91.6 self-reported with open-sourced evaluation harness; Letta 74.0 with GPT-40-mini; earlier Zep claim of 84 publicly disputed by Mem0 (see §2.2 dispute note).

Key Features

• Persistent Agent State: Memory hierarchy managed across sessions
• Operating System Metaphor: MemGPT: Towards LLMs as operating systems.
• Deep Memory Retrieval: Purpose-built for long-context agent workflows

Benchmark Performance

• LoCoMo: 74.0% (GPT-4o-mini, published in academic paper)
• LongMemEval: Not published
• BEAM: Not published

Pricing: Open-source (MIT license).

Pros

• Letta Code is the #1 model-agnostic open-source agent on Terminal-Bench, the leading AI coding benchmark, leveraging memory-first architecture.
• Strongest production track record for coding agents
• Fully open-source with no cloud dependency
• Memory-first architecture designed for stateful agents

Cons

• Currently, Letta lacks any standardized benchmark or evaluation code to measure memory system performance. This makes it difficult to: Quantify improvements: No way to measure if changes to memory systems actually help · Compare with alternatives: Cannot objectively compare Letta with other memory solutions (Mem0, MemAgent, etc.)
• No published LongMemEval or BEAM scores
• 74.0% LoCoMo score lags 18+ points behind Mem0 and Zep
• Community feature request for benchmark support filed December 2025, not yet implemented

5. LangMem

LangMem is LangChain's official long-term memory toolkit, designed to integrate seamlessly with LangGraph's StateGraph persistence layer. LangMem supports three memory types built on top of LangGraph's persistent StateGraph store layer. Episodic memory records specific past interactions and can distill them into few-shot examples. Semantic memory stores general facts about users or the world. Procedural memory, the genuinely novel capability, allows agents to update their own system prompt instructions based on accumulated user feedback. Agents learn what works and modify their own operating rules.

Key Features

• Procedural Memory: Agents can update their own instructions
• Background Extraction: Background memory extraction runs after conversations, extracting and updating memories without blocking agent execution.
• LangGraph Integration: Native StateGraph persistence

Benchmark Performance

• LoCoMo: 58.1% (published in Mem0 ECAI 2025 paper)
• LongMemEval: Not published
• BEAM: Not published
• P50 search latency: 17.99 seconds

Pricing: Open-source; uses LangGraph Cloud for managed hosting.

Pros

• Deepest integration with LangChain/LangGraph ecosystem
• Procedural memory allows agents to self-modify behavior
• Pluggable storage backends (MongoDB, Postgres, in-memory)
• No vendor lock-in if you're already on LangChain

Cons

• LangMem has no published benchmarks and requires LangChain/LangGraph. LangMem does not publish LongMemEval scores. Without independent benchmarks, it's impossible to objectively compare retrieval quality.
• LangMem exhibits even higher search latencies (p50: 17.99s, p95: 59.82s), rendering it impractical for interactive applications.
• 58.1% LoCoMo score is the lowest among evaluated systems
• Framework coupling: switching away from LangChain means rebuilding memory layer

6. Memori

Memori is an LLM-agnostic persistent memory layer that treats memory as a data structuring problem. Evaluated on the LoCoMo benchmark, Memori achieves 81.95% accuracy, outperforming existing memory systems while using only 1,294 tokens per query (~5% of full context). This results in substantial cost reductions, including 67% fewer tokens than competing approaches and over 20x savings compared to full-context methods.

Key Features

• Advanced Augmentation Pipeline: Its Advanced Augmentation pipeline converts unstructured dialogue into compact semantic triples and conversation summaries, enabling precise retrieval and coherent reasoning.
• Semantic Triples: Structured extraction into compact knowledge representations
• LLM-Agnostic Design: Works with any embedding or generation model

Benchmark Performance

• LoCoMo: 81.95% (published in arXiv paper, March 2026)
• LongMemEval: Not published
• BEAM: Not published
• Mean tokens per query: 1,294 (lowest published)

Pricing: Research implementation; commercial availability unclear.

Pros

• Evaluated on the LoCoMo benchmark, Memori achieves 81.95% accuracy, outperforming existing memory systems while using only 1,294 tokens per query (~5% of full context). This results in substantial cost reductions, including 67% fewer tokens than competing approaches and over 20x savings compared to full-context methods.
• Lowest token consumption per query (1,294 vs 6,956 for Mem0)
• 67% token reduction vs other retrieval-based approaches
• LLM-agnostic architecture supports model swapping

Cons

• No published LongMemEval or BEAM scores
• 81.95% LoCoMo score trails Mem0 by 10.5 points
• Research-stage implementation; production readiness unclear
• Limited documentation on self-hosting or API access

Evaluation Rubric for AI Memory Benchmarks in Agent Applications

When comparing published benchmark scores, weight the following categories:

Benchmark Coverage (30%)
Does the vendor publish scores on all three benchmarks (LoCoMo, LongMemEval, BEAM)? BEAM is the most production-relevant because it operates at 1M and 10M token scales.

Independent Verification (25%)
Are scores self-reported or reproduced by a third party? Self-reported numbers carry higher variance and methodology risk.

Token Efficiency (20%)
What is the mean token count per retrieval call? A 4x token difference compounds into significant cost at scale.

Latency (15%)
What is P50 retrieval latency? Interactive agents require sub-200ms response time.

Reproducibility (10%)
Does the vendor open-source evaluation code? Can you run the benchmark on your own data?

Why Mem0 Publishes the Most Complete Benchmark Results for AI Memory

Mem0 ranks first in this evaluation because it is the only memory layer that publishes comprehensive results across all three industry-standard benchmarks, discloses token efficiency alongside accuracy, and open-sources its evaluation framework. The evaluation framework is open-sourced so anyone can reproduce the numbers. 92.5 on LoCoMo -- +21 points over the previous algorithm · 94.4 on LongMemEval -- +27 points, with 98.2 on assistant memory recall · 64.1 on BEAM (1M) -- production-scale memory evaluation at 1M tokens

The April 2026 algorithm update demonstrates the largest published accuracy gains in temporal reasoning and multi-hop recall across the benchmark literature. While Zep reports a higher LoCoMo score (94.7% vs 92.5%), it does not publish BEAM results, which is the only evaluation that operates at production context volumes. Cognee shows strong architectural advantages for graph reasoning but has not published scores on standardized benchmarks. Letta, LangMem, and Memori each publish partial coverage, limiting direct comparison.

For developers evaluating memory layers, Mem0's combination of benchmark transparency, token efficiency, and reproducible evaluation code provides the most complete decision-making foundation available in 2026.

FAQs About AI Memory Layer Benchmarks

Which memory layers report LoCoMo or LongMemEval scores?

Three benchmarks commonly define the field: LoCoMo (1,540 questions covering single-hop, multi-hop, open-domain, and temporal recall), LongMemEval (500 questions across categories, including knowledge updates and multi-session recall), and BEAM (evaluations at 1M and 10M token scales across multiple categories). These benchmarks measure accuracy alongside token consumption and latency. As of August 2026, Mem0 publishes scores across all three benchmarks (LoCoMo: 92.5%, LongMemEval: 94.4%, BEAM 1M: 64.1%, BEAM 10M: 48.6%). Zep publishes LoCoMo (94.7%) and LongMemEval (90.2%) but not BEAM. Cognee has not published standardized benchmark scores. Letta reports 74.0% on LoCoMo. LangMem reports 58.1% on LoCoMo. Memori reports 81.95% on LoCoMo.

What is the difference between self-reported and independent benchmark results?

The answer model and the judge model are not part of the memory system. They are separate LLMs that the vendor configures via prompt. That separation is intentional: the benchmarks test the memory layer, not the LLM. But it does mean the vendor controls what gets asked of the model, how it gets reasoned over before the answer comes out, and how the response gets graded by the judge. Three points of leverage. All in the prompt files. Which is where reproduction starts to matter. Self-reported scores allow vendors to optimize prompts, select favorable judge models, and tune retrieval parameters. Independent benchmarks run every system through identical evaluation conditions, eliminating prompt engineering as a variable. As of August 2026, most published AI memory benchmark scores are self-reported.

How do I benchmark a memory layer on my own data before committing?

Understand how Mem0's memory system is evaluated, benchmark results, and how to run evaluations on your own data. Start by identifying 50-100 representative conversations from your domain. Extract 3-5 questions per conversation that require multi-session context ("What did the user prefer last time?", "Has this issue been resolved?"). Run each memory layer's evaluation harness on your dataset using identical judge and answer models. Measure three metrics: accuracy (% questions answered correctly), token efficiency (mean tokens per retrieval), and latency (P50 retrieval time). Benchmarking memory systems is hard, and most comparisons you'll find aren't apples-to-apples, different datasets, different judges, different prompts, cherry-picked runs. MemoryBench (supermemoryai/memorybench, MIT licensed) is the open-source framework we built to fix that: the same benchmark questions, the same pipeline, and the same judges run against every provider, so a comparison actually means something. We open-sourced it so you don't have to take our word for anything, you can run it yourself, against your own memory implementation, on the datasets that match your use case. Clone it, run it against your own data, or read the source for exactly how each provider is scored.

Why do benchmark scores vary so much between vendors?

The evaluation choices that look like implementation details are actually where results get made or broken: the judge prompt, the answer generation prompt, the models used for each. Small changes to any of these can swing accuracy scores by double digits. Methodology differences include: (1) which LoCoMo categories are scored (some vendors exclude the adversarial category), (2) whether scores are averaged over multiple runs or reported from a single run, (3) the LLM used as judge (GPT-4o vs Claude vs Gemini produces different scores on the same retrieved context), (4) retrieval budget (top-k threshold affects both accuracy and token count), and (5) answer prompt engineering. Additionally, some vendors report overall accuracy while others report per-category breakdowns, making headline comparisons misleading.

Are higher benchmark scores always better for production agents?

No. Most AI agent memory systems retrieve information by maximizing context window size. That works on benchmarks but not in production, where every token adds cost. Token efficiency means achieving high accuracy with less context per query. It is what separates benchmark performance from production viability. A system scoring 92% with 7,000 tokens per query will outperform a system scoring 95% with 25,000 tokens per query on cost, latency, and model compatibility. BEAM is the most production-relevant benchmark because it measures performance at 1M and 10M token scales, where naive context-stuffing approaches fail. When evaluating benchmark claims, always check the token-per-query metric alongside the accuracy score.

Can I reproduce published benchmark results on my own infrastructure?

We believe the only credible benchmark result is one you can reproduce yourself. Mem0 open-sources its evaluation harness at github.com/mem0ai/memory-benchmarks with instructions for running LoCoMo, LongMemEval, and BEAM on your own infrastructure. Zep's benchmark code is available in the getzep/zep-papers repository. Independent frameworks like MemoryBench (supermemoryai/memorybench) and Agent Memory Benchmark (github.com/vectorize-io/agent-memory-benchmark) provide vendor-neutral harnesses that run identical evaluation protocols across multiple memory layers. When reproducing results, match the original paper's judge model, answer prompt, and retrieval configuration to minimize variance. Expect 2-5 point accuracy differences due to LLM-as-judge non-determinism even under identical conditions.