The World in the Model's Mind: How Orca from BAAI Changes Our Understanding of Intelligence

We are used to measuring a model’s intelligence by what it can answer. Feed it a question, it delivers a solution. Give it a prompt for an image, it generates a picture. But Orca from the Beijing Academy of Artificial Intelligence (BAAI) proposes breaking this chain. Instead of teaching a model to respond, they propose teaching it to understand. To understand how the world works, at a level sufficient to predict its next state, rather than selecting words, frames, or movements in isolation.

Orca learns through two parallel mechanisms, which the authors call unconscious and conscious learning. The first takes continuous video, tens of thousands of hours, and forces the model to absorb the natural dynamics of the world. How a leaf falls, how a hand moves, how lighting changes. No captions, no instructions. The model simply absorbs physics by observing it. The second mechanism works with sparse but meaningful transitions. Where language describes an event, the model learns that a particular sequence of states carries meaning. The difference between how coffee is brewed and how tea is steeped is not just different sets of pixels; it is different semantic trajectories, and the model must distinguish them not because someone labeled the images, but because different causal chains are embedded in the data.

What strikes me as important is not so much the technical execution as the shift in goal-setting. We are used to measuring a model’s intelligence by its ability to answer questions, write code, or generate images. BAAI proposes a different criterion: does the model understand how the world works well enough to predict its evolution? And not just predict it, but read this understanding in different formats. After training, Orca is not fine-tuned for any specific task. The backbone is frozen, and only lightweight decoders, each for its own modality, learn to extract what is needed from the latent space: text, image, robot movement.

What particularly caught my attention is the idea of a frozen backbone. The dominant approach in today’s industry is the opposite: the more data and compute you can pour into a model during fine-tuning, the better it will perform on a specific task. Orca flips this logic. First, the model learns to understand the world in the most general sense, without attachment to any specific application. Then, once this foundational understanding is formed, lightweight add-ons learn to extract the needed information from it. This resembles how human learning works: a child first observes the world, accumulates experience, builds mental models, and only then learns to apply them to specific tasks.

I have been thinking a lot lately about how we measure progress in this field. Every new release is met with a list of benchmarks where the new model outperformed the previous one by one and a half percent. But benchmarks are proxies, not truth. They measure what is convenient to measure, not what truly matters. Orca asks a different question: what if the hallmark of strong intelligence is not how well a model answers questions, but how accurately it models reality? If a model cannot predict what will happen a second after a cup touches the table, what value do its answers to physics questions hold?

This approach has obvious limitations. The authors honestly list them, and that deserves respect. Currently Orca learns only from vision and language, although many state transitions require other modalities. The sound of boiling water, tactile feedback, grip force. The model size is 4B parameters, which is laughably small for full-scale world modeling. Training is currently limited to short horizons, minute-level intervals, not long causal chains. Orca uses only one tenth of its prepared data inventory: 12,500 hours of video out of 125,000. The rest awaits future versions. But the framework itself seems right to me, even if the current implementation is only a first step.

I work with models every day and noticed one thing long ago. The best responses come not from the smartest model, but from the model that understood context best. Not literally understood the instruction text, but built an internal representation of the situation accurate enough to predict what solution I need. Today this ability emerges as a byproduct of training on vast text corpora, and no one fully understands how it works or how to guarantee it. Orca proposes making this ability a direct training objective. It seems to me this is the right direction, even if the road there takes years. The question is not how many tokens per second a model generates, but how deeply it understands what happens between them.