Intelligence Before Language
Lately, I have been reading Max Bennett’s *A Brief History of Intelligence*. The book reopened a question for me: when we talk about “intelligence,” what exactly are we talking about?
Today, it is easy to recognize intelligence through language, logic, and problem-solving. Someone who is knowledgeable, articulate, and rigorous in their reasoning is usually considered intelligent. These also happen to be the areas in which large language models have progressed most rapidly, so it is only natural that we experience AI’s intelligence primarily through its answers.
But in the evolutionary history Bennett traces, language is a very late arrival.
Long before language, living organisms could already distinguish what in their environment was beneficial from what was harmful. They could approach opportunities and avoid danger, learn from the consequences of their actions, simulate possible outcomes before acting, and gradually infer the intentions of others.
Bennett presents this history as a simplified model of five breakthroughs: steering, reinforcing, simulating, mentalizing, and speaking. His original account of the framework is explicitly a first approximation rather than a complete theory of brain evolution.
From this perspective, intelligence begins with how a living organism evaluates its environment and acts within it—not with language.
What deserves attention? What is worth pursuing? Is an immediate reward worth the risk? Should an organism persist, or stop before it is too late? These questions have no universal answers outside a specific situation, yet they have always accompanied survival and action.
Emotion as a Value System
Emotion—and the pull of approach and avoidance—is part of this calculus.
Fear increases the weight of danger in a decision. Anticipation makes an outcome feel more worth pursuing. Satisfaction helps determine when a course of action should end. These signals can sharpen judgment or distort it, but they are not irrelevant noise sitting outside reason. They help determine what matters, turning perception, memory, and reasoning into actual choices. Decades of research likewise describe emotion as a pervasive driver of judgment and choice—sometimes harmful, sometimes beneficial. Lerner et al., *Emotion and Decision Making*
Nor is approach and avoidance merely a mechanical reflex. Even very simple forms of life must weigh competing signals. Food draws an organism closer; danger drives it away; hunger changes how much risk it is willing to tolerate. Here we already see one of intelligence’s most basic functions: assigning value to possible outcomes and choosing an action accordingly.
An example Ilya Sutskever discussed in an interview points in a similar direction.
He recalled reading about a patient who lost normal emotional processing after suffering brain damage. The patient could still speak fluently and apparently perform well on certain cognitive tests, yet struggled to make ordinary decisions. Choosing which pair of socks to wear could take hours, while decisions in everyday and financial life became increasingly poor.
Language remained intact. Analytical ability had not disappeared. What had broken was the ability to turn analysis into effective choice.
Ilya suggested that emotion may regulate something like a “value function.” It does more than tell us which options exist. It gives possible outcomes different weights, allowing us to judge whether we are doing better or worse—and to change course in time. Ilya Sutskever in conversation with Dwarkesh Patel
Emotion, approach and avoidance, and valuation were part of intelligence long before language.
A system may possess enormous knowledge. It may enumerate every available option and accurately describe the consequences of each. But it must still determine what matters in the present situation, what is worth pursuing, when to abandon a course of action, and how failure should change what it does next.
Why LLMs Do Not Make AGI Inevitable
This is why I remain uncertain about the path from LLMs to AGI.
The linguistic capabilities of LLMs are extraordinary, and language training can clearly produce abilities that extend beyond language. Language contains immense amounts of information about the world, causality, the consequences of actions, and human intentions. To predict language, a model may need to learn some of the structures beneath it.
But there is still a considerable distance between saying that language training can produce some non-linguistic abilities and saying that LLMs will inevitably develop into AGI.
As a system becomes better at talking about fear, does it also become better at recognizing danger in a real situation? When it can explain a failure, does that failure meaningfully change its next decision? When its environment changes, can it reevaluate what matters—or does it simply continue pursuing its original objective?
Machines do not need to experience emotion in the same way humans do. They may implement valuation, learning from experience, and action selection through entirely different mechanisms. What remains uncertain is whether these abilities will emerge naturally as language models scale, or whether they will require new training methods, learning mechanisms, and system architectures.
My current view is this: LLMs do not necessarily lead to AGI.
“Not inevitable” does not mean “impossible.” Systems built around LLMs may well become an important path toward general intelligence. But progress in language alone is not enough to demonstrate that every other component of intelligence will arrive automatically.
A New Kind of Tool
At the same time, I am deeply optimistic about LLMs. They are already remarkably malleable and extensible tools.
LLMs are expanding the boundary of what a tool can be.
Traditional tools usually require a person to determine the steps in advance and then operate a set of predefined functions. An LLM can begin with a relatively ambiguous goal and participate in clarifying the problem, organizing information, and adjusting the path forward. The same model can move among writing, research, analysis, and programming, while connecting to memory, knowledge bases, and external tools.
Computer use makes this shift especially visible.
An AI can operate a browser and a computer, inspect webpages, use applications, organize files, and choose its next step based on what happens. Given a research objective, it can decide how to search, which pages to open, what information to extract, and how to turn its findings into a document. When a website changes or the available evidence proves insufficient, it can revise its original approach.
One tool is beginning to use another with an extraordinary degree of freedom.
Historically, a tool carried out an action. Now, it can take on an entire process—one that requires observation, judgment, and adjustment. This is already a profound change. Its value does not depend on whether, or when, AGI arrives.
Generalization Is Still the Open Question
Using many tools flexibly and possessing a more complete form of intelligence are not the same thing. The former expands the range of tasks a system can perform. The latter also concerns how the system evaluates a situation, accumulates experience, revises itself, and reliably transfers what it has learned to unfamiliar environments.
In the same interview, Ilya also focused on generalization and continual learning. Current models still generalize far less robustly than humans. People do not learn every job they will ever perform before entering society. We begin with a foundation of abilities, then continue to grow through experience and trial and error in particular environments. More capable AI may need to undergo a similar process. Sutskever on generalization and continual learning
I can therefore be optimistic about LLMs while remaining skeptical that they inevitably lead to AGI. There is no contradiction between the two.
The optimism comes from a change that is already underway: AI is redefining what tools can do. The skepticism comes from a question that remains open: how does a system that understands and generates language develop the ability to value outcomes, learn continuously, and act reliably?
Language Is Not the Whole of Intelligence
Finally, I also recommend A Brief History of Intelligence.
At a higher level of abstraction, the evolution of intelligence is also the story of ever-expanding sources of learning: from the consequences of one’s own actions, to simulations inside the mind, to the experiences and imaginings of other minds. Language allows those experiences to travel across individuals and generations, ultimately creating a culture that can continue to accumulate.
But language is not the whole of intelligence. The mechanisms we often call “emotion” or “instinct” also participate in how living beings evaluate their circumstances: what to approach, what to avoid, which outcomes matter, and when it is time to act.
References
- Max Bennett, *A Brief History of Intelligence*.
- Max Bennett, “Five Breakthroughs: A First Approximation of Brain Evolution From Early Bilaterians to Humans”, Frontiers in Neuroanatomy, 2021.
- Jennifer S. Lerner et al., “Emotion and Decision Making”, Annual Review of Psychology, 2015.
- “Ilya Sutskever — We’re Moving From the Age of Scaling to the Age of Research”, interview with Dwarkesh Patel, 2024.