19 Q UA RT E R _ 0 2 _ 2 0 2 6 - - - - - - » Outlook an understanding of the physical world. What are the applications of world models? Companies that produce video content (for example, media, advertising, social networks) and interactive environments (for example, video games, 3D training) benefit from world models that can realistically predict how the physical world will react to actions and movement. Com panies that build physical products that they want controlled by AI—for example, autonomous vehi cles, industrial machines, and robots of various kinds—can also embed world models into their products. World models can also be useful for training other AI models that are meant to operate (that is, control machines) in the physical world. How can world models be used for AI training? There is a technique for training AI models called reinforcement learning: The model being trained is given a goal, and the actions it takes that result in progress toward that goal are “rewarded.” Models being trained could be set loose in the real world, but it’s much easier, faster, and safer to use a world model to simulate what might happen in the real world when a model in training takes an action. Would you want to train autonomous vehicles by just setting them loose on city streets? (Uh, never mind.) The researcher often considered the father of AI reinforcement learning, Richard Sutton, was recently interviewed by the prolific (and enjoyably geeky) AI podcaster Dwarkesh Patel. The conver sation inspired a lot of commentary (which carried over into a follow-up podcast with AI researcher and Eureka Labs founder Andrej Karpathy), partly because it surfaced a fundamental limitation of classic transformer-based foundation models, including LLMs (which Sutton calls an AI “dead end”): They do not learn from their “experiences” in the way that human beings and animals are always continually learning. In other words, they have no long-term memory. To change an LLM’s behavior on a long-term basis, you have to take them “back to the AI garage” and retrain/fine-tune/modify the connection weights of their neural networks. This brings up a pet peeve of mine about agents [RANT ON]: I’ve seen a lot of descriptions of AI agents that assert they learn autonomously. But if they’re constructed with LLMs, this is not true. LLMs will react within their context windows based on the dialogue you are having with them (this is the “in-context learning” mentioned in the Dwarkesh Patel podcast with Andrej Karpathy noted earlier), but once you’re in another context window—that is, you’ve started a new conversation with an LLM—it’s as if you just restarted the model out of the box; it forgets everything from the previous dialogue. It hasn’t learned anything from your prior interactions (although developers of products such as Claude, ChatGPT, and Gemini have cre ated hacks like adding connections to databases to try to create some kind of memory; results may vary). The strength of the connections, or “weights,” in a typical artificial neural network do not change after the initial training, unlike those in a real brain [/RANT OFF]. The process of scientific discovery also requires an understanding of the physical world. In fields such as life sciences and chemicals, companies are developing new products by using deep learn ing surrogate models that simulate aspects of the physical world and thus can be described as a type of world model. Our other research has shown that these and related AI techniques could help to dou ble the pace of R&D in these and similar fields. The “AI for science” realm is buzzing, with start-ups and established AI leaders digging in. World models are still early, imperfect, and evolving—but they point toward systems that can reason about cause and effect in the physical world. If that trajectory holds, we may look back on today’s language models as the prologue to a much more interesting story. Michael Chui is a senior fellow in McKinsey’s Bay Area office. This column is part of his Quantum of Solis series on AI.