Benchmarking Robot Memory Under Interference

Contributions

• A cross-session interference benchmark. We extend RoboMME with a controllable history buffer: a relevant prior demonstration followed by k ∈ {0,1,3,7} unrelated sessions, turning memory distance into a measured variable.
• A complete, released result grid. Nine RoboMME task families × five history conditions × 50 episodes for each of eight released memory systems, plus the π_0.5 baseline with no history: 18,450 rollouts.
• A measurable failure mode. Perceptual memory delivers large near-session gains (FrameSamp-Modul: +27 pp at k0) that erode steadily under interference (−26 pp by k7), while recurrent variants stay near baseline. Systems that look strong at short range can collapse across sessions.
• The decay is recoverable. Retrieval-FrameSamp-Modul, which chunks the history with visual embeddings and passes only the matching chunk to the policy, stays near 45% under k7 interference where FrameSamp-Modul falls to 19.3%, and recovers the k0 rate on seven of the nine families.

The Benchmark

Every query episode is run through the same policy interface, varying only the external history buffer. The relevant lesson, a prior demonstration of the task, always appears first; the history conditions differ in how many unrelated sessions are inserted before the query reads it.

Why these nine tasks. The protocol requires the task-relevant information to live in a self-contained prior session that can be lifted into the history buffer. We therefore use the nine RoboMME tasks that deliver this information as a prior demonstration video, distinct from execution; the remaining tasks embed their cue inside a single episode and have no separable prior session to place under interference.

Why unrelated distractors. Distractor sessions are drawn from different task families, so they cannot inject contradictory same-family facts. The benchmark therefore measures interference from genuinely unrelated robot experience, the deployment-like failure mode, rather than constructed wrong priors.

Tasks

Nine task families from RoboMME, each presenting a prior demonstration the policy must use during a separate execution episode. They span three memory types.

Task definitions and assets are from RoboMME (Dai et al., 2026).

Memory Systems

We run RoboMME's released memory-augmented variants through the benchmark. Each pairs a memory representation (frames, tokens, or hidden states) with a mechanism for integrating that memory into the base π_0.5 policy: Context concatenates memory tokens with the input, Modulator conditions the policy via adaptive LayerNorm, and Expert routes memory through a dedicated expert with block-wise causal attention. The perceptual variants (FrameSamp, TokenDrop) sample or drop visual-history tokens; the recurrent ones (TTT, RMT) keep a compressed hidden state. The baseline is the π_0.5 backbone with no memory.

✓ evaluated: eight memory variants plus the π_0.5 baseline (nine systems).
– not evaluated: RMT and TTT-Modulator had no released checkpoints at evaluation time.

Results

1 Relevant memory helps — when it is near

At k0, with only the relevant session in history, perceptual memory lifts success well above each system's own no-history floor: FrameSamp-Modul from 18.2% to 45.3% (+27.1 pp) and TokenDrop-Modul from 17.1% to 35.3% (+18.2 pp), against the 17.3% π_0.5 baseline.

Full grid

Success rates across all nine families.

Adding Retrieval

The benchmark systems sample the whole history down to a fixed set of frames, so once other sessions are present, only a few of those frames come from the relevant demonstration, which is the likely cause of the performance drop. To fix this, Retrieval-FrameSamp-Modul adds a retrieval step to FrameSamp-Modul: it chunks the history using visual embeddings, finds the chunk that matches the current view, and passes only that chunk on. The full frame sampling budget then goes to the demonstration.

This holds the k0 success rate through k7 on seven of the nine families. Where FrameSamp-Modul falls from 45.3% at k0 to 19.3% at k7, Retrieval-FrameSamp-Modul stays near 45% the whole way.

Retrieval is easy in this setup. The matching demonstration is always present and shares the query's task family, so a visual match reliably finds it.

Reproducibility & Artifacts

Everything needed to reproduce the grid and build on the protocol is released.

• Paper — the full method and protocol on arXiv.
• GitHub repository — code, analysis scripts, and figures.
• Canonical rollout CSV — all 18,450 rollouts, the source of truth.
• Main success table — per-system rates with confidence intervals.
• Results summary — headline numbers and effects.

Citation

@misc{rathi2026robommeinterference,
  title         = {Benchmarking Robot Memory Under Interference},
  author        = {Rathi, Soumil},
  year          = {2026},
  eprint        = {2606.22338},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2606.22338}
}

This benchmark builds on RoboMME; please cite it as well.

@article{dai2026robomme,
  title  = {RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies},
  author = {Dai, Yinpei and Fu, Hongze and Lee, Jayjun and Liu, Yuejiang and
            Zhang, Haoran and Yang, Jianing and Finn, Chelsea and Fazeli, Nima and Chai, Joyce},
  journal = {arXiv preprint arXiv:2603.04639},
  year   = {2026}
}