AI Agents Forget Everything — Here Is What Teams Are Actually Doing About It
The complaint comes up in almost every team retrospective once AI agents go into production. Not a technical complaint, not a model accuracy complaint. Something simpler: the agent forgets.
In customer service, it shows up as the bot that asks for your account number again, three minutes into a conversation you already verified yourself at the start of. In software development it shows up as the coding assistant that confidently suggests a refactor, ignoring a convention it established forty minutes and thirty files ago. In long-running research workflows it shows up as the agent that starts strong, drifts slowly, and by the end is solving a problem that was only tangentially related to the one you actually gave it.
These are not separate issues. They are the same issue wearing different costumes: memory failure in autonomous AI systems is becoming the primary complaint field for teams deploying agents at scale, and the solutions emerging from research labs are finally getting serious enough to talk about.
The token cost problem nobody wanted to talk about
Context windows keep growing. Models that could handle 128,000 tokens a year ago now handle millions. The industry narrative around this has been straightforward: bigger windows mean more memory, more memory means better performance. That framing is technically accurate and practically misleading.
Bigger windows do not solve the memory problem. They relocate it. A model with a million-token context window will dutifully fill it, and then fill it some more, and by the time you are running production queries at scale, the token cost per interaction has become a line item that makes finance squirm. Some deployments are processing millions of tokens per query. At scale, that math stops working even if the model performance is genuinely better.
The research community started calling this out more directly around mid-2025. Not because the problem was new, but because the deployments finally got large enough that the cost was impossible to hide inside a promising demo. When you run a thousand queries a day, and each one burns through a million tokens, the infrastructure bill is not an abstraction. It is a conversation with your CFO.
Two approaches have emerged as the serious candidates for solving this at the engineering level, not just the research paper level.
MRAgent: reconstruction instead of retrieval
The first direction abandons the idea of storing everything and handing it to the model every time. MRAgent and similar reconstruction-based approaches work on a different premise: instead of maintaining a perfect record of everything that has happened, analyze what the current task actually needs and build the relevant memory on demand.
The numbers behind this are what make it worth taking seriously. Teams working with MRAgent-style architectures report around 27x reduction in memory tokens compared to naive full-context approaches. Runtime drops by roughly half. The task performance does not collapse from the compression; in many cases it holds or improves, because the agent is no longer sorting through decades of irrelevant transcript to find the three things that actually matter for the current step.
The conceptual shift underneath this is important. A task that has been running for fifty steps does not need a complete record of every intermediate thought. It needs access to the decisions that shaped where it is, the patterns it has established, and enough context to make step fifty-one coherent with everything that came before. MRAgent tries to identify what those are before retrieval rather than after.
This is not a solved problem. Reconstruction quality varies significantly depending on task type, and the mechanisms for determining what matters in a long-running session are still being refined. But the approach has moved from theoretical interest to active production use in a way that was not true twelve months ago.
LangMem: quality over quantity in what gets stored
The other major direction takes the opposite angle. Rather than focusing on retrieval efficiency, LangMem focuses on what gets stored in the first place.
The target is around 118,000 meaningful tokens per query, but the emphasis is on the word meaningful. LangMem-style memory systems push back hard on the idea that a transcript is a memory. A transcript of a meeting tells you what was said. It does not tell you what was decided, why the decision was made, or what constraints shaped the options. LangMem tries to store the latter: a working model of the agent’s state, goals, constraints, and the patterns it has learned to follow.
The distinction matters more than it sounds. Agents running on transcript-based memory can retrieve everything that was said and nothing of what was understood. They surface the right information at the wrong abstraction level, or they surface information that was relevant three sessions ago but is now stale in ways the retrieval system cannot detect. Systems built on explicit semantic models of agent state can reason about relevance and freshness in ways that transcript retrieval cannot.
This is harder to implement than it sounds. Building a reliable model of what an agent understands about its own task requires the system to reason about its own reasoning, which is a harder problem than storing a transcript. But the teams working on this are producing results that justify the extra complexity for agents running complex multi-step tasks over extended timeframes.
What enterprise teams are actually dealing with
The research abstractions matter, but what they matter for is the day-to-day reality of deploying agents in environments where failure has real costs.
The customer service case is the most visible. A bot that forgets context mid-conversation is not just annoying; it actively damages trust in ways that are hard to recover. Users who have been asked to repeat information they already provided once start the interaction already frustrated, before the actual problem has even been addressed. For companies that have invested in building a coherent brand experience through their digital channels, an agent that creates this specific frustration is a significant brand liability, not just a technical problem.
The coding assistant case is subtler but equally real. Development teams using AI agents for long refactoring sessions, test generation, or codebase-wide convention enforcement run into a specific failure mode: the agent starts applying the wrong conventions because it has lost track of which version of the project it is working against. The cost is not just incorrect code; it is the time spent code reviewing output that should never have been generated, and the reduced trust in the tool that follows.
The research synthesis case is where memory failure gets most expensive. Multi-day research projects that involve dozens of sources, hundreds of citations, and complex chains of reasoning are exactly where agents have the most potential value and the most spectacular failure modes. An agent that starts a second session with no memory of what was established in the first is not a research tool. It is a session-long autocomplete.
The engineering discipline that is forming
MRAgent and LangMem are names for research directions, not products. But the ideas behind them are filtering into production systems at a pace that is worth tracking if you are building in this space.
What is happening is the emergence of memory management as a first-class engineering discipline in the AI stack. Not as an afterthought. Not as a problem that gets solved by buying a bigger context window. As an actual discipline with actual tradeoffs, actual implementation complexity, and actual competitive implications for teams that get it right versus teams that do not.
The teams that figured this out early are writing the blog posts about it now. The teams that are about to figure it out are the ones generating the support tickets that the first group is writing about.
The 27x token reduction from reconstruction-based approaches is the headline. What it means underneath is that a new engineering layer is forming between the model and the application: the memory management layer. Teams that treat this as a design problem rather than a configuration problem will move faster, cost less at scale, and produce agents that are more reliable in the ways that actually matter to users.
The transition from research question to infrastructure question to business question happens faster than most people expect in this industry. We are somewhere in the middle of that transition for AI agent memory. The teams that are paying attention to the infrastructure question now are going to be the ones who look prescient in eighteen months.
Start by assuming the built-in context is not enough for production. It probably is not. And the cost of finding out under load is higher than finding out in the design phase.
