Google Research has published a "sleep framework" that lets large language models keep learning after they are deployed. Modern LLMs are largely frozen once training ends: their knowledge stops at a fixed cutoff date, and while in-context learning can add facts temporarily, all of it is forgotten the moment the context window closes. Simply training on new data is not a fix either, because continual parameter updates trigger catastrophic forgetting, where learning something new degrades what the model already knew.
The framework borrows its structure from neuroscience. A model alternates between active interaction, where new input is stored in short-term memory, and an offline sleep cycle made of two stages. During consolidation, fragile short-term context is compressed into long-term parameters using low-rank structures and pre-allocated Mixture-of-Experts memory blocks. During dreaming, the model generates its own practice examples while random expert routing and reinforcement learning refinement widen the set of updates it explores, reinforcing the most useful ones.
Google reports broad gains over in-context learning and supervised fine-tuning. On continual translation into unseen languages such as Manchu and Kalamang, the sleep approach held performance steady where standard baselines suffered severe forgetting. On the BABILong long-context benchmark it scaled to 10 million tokens with near-perfect accuracy, beating GPT-4 and Titans. It scored 48.9% on single-task SQuAD and 46.2% on continued pre-training, ahead of SEAL. On math benchmarks it reached 79.2% on AIME 24, 69.0% on AIME 25 and 46.1% on HMMT 25. Google frames the result as a step toward AI systems that keep improving after deployment rather than freezing at launch.