A Good Guess Isn’t a Mind

A chatbot predicts what your child wants, remembers the name of the family dog, and produces exactly the reassuring sentence the moment calls for.

It can feel uncanny. Maybe even thoughtful.

But a good guess is not a tiny person peering through the screen. Prediction and understanding can travel together, but they are not the same ticket.

I say this as someone who recently bought hand puppets to demonstrate false-belief tasks and instead spent several evenings testing them on a cat. The cat failed to cooperate, which I refuse to interpret as evidence that he lacks a theory of mind. He may simply lack respect for peer review.

AI is getting better at guessing us

The strongest version of the “it’s just prediction” argument is no longer especially comforting. Prediction can be astonishingly powerful.

Consider Centaur, a foundation model adapted using Psych-101, a collection of human behavioral experiments. It was designed to predict and simulate human behavior across varied cognitive tasks (Binz, 2025). That matters. A model that captures patterns across many experiments may become a useful scientific instrument, not merely an autocomplete wearing a lab coat.

Still, success at predicting a choice does not establish that the model experiences indecision, recognizes another person as a conscious being, or understands why the choice matters.

A weather model can predict rain without getting wet. The analogy is imperfect—people are not clouds, despite what family road trips suggest—but the distinction holds. Accurate output tells us that a system has captured useful regularities. It does not, by itself, tell us what kind of inner process produced them.

This is where AI discourse usually sprints toward a cliff in one of two directions. One camp says, “It talks like us, therefore it thinks like us.” The other says, “It predicts patterns, therefore nothing interesting is happening.” Both conclusions are doing more work than the evidence.

The boring middle is better: sophisticated behavioral modeling is real, useful, and not a consciousness detector.

Children learn minds through relationships

Human social understanding is also inferential. Children cannot directly inspect another person’s beliefs. They watch, listen, compare expectations with behavior, and gradually learn that someone else can know less, want something different, or hold a mistaken belief.

Computational accounts describe this development partly as a kind of probabilistic inference. But the training environment is not a static archive of sentences. It is social life: conversations with caregivers, shared experiences, misunderstandings, repairs, facial expressions, and the recurring discovery that other people have perspectives of their own. Jara-Ettinger (2025) argues that social experience and parent-child conversation help shape children’s developing ability to reason about minds.

That difference matters. A child’s theory of mind develops inside relationships where predictions have consequences. If Dad thinks the toy is in the cupboard when it is actually under the couch, the child can notice the mistake, explain it, correct it, and see what happens next. Knowledge is tied to a particular person, history, goal, and world.

An AI system may produce a convincing explanation of Dad’s false belief. That performance deserves attention. It still does not prove the system regards Dad as someone with an inner life—or has one itself.

Learning is more than changing connections

There is another reason to resist easy equivalence between brains and models. Even in neuroscience, “the connections changed” is not a complete explanation of learning.

Lisberger (2024) argues that synaptic plasticity is the currency of learning, but not the whole economy. Behavioral learning emerges through interacting plasticity sites, recurrent circuits, and coordination across levels of a neural system. In other words, finding an adjustable connection does not mean we have found the mind.

Artificial networks also adjust connections, of course. That is a meaningful similarity, not a magic identity card. A piano and a laptop both contain keys; this does not make the laptop a piano. Though, annoyingly, the laptop can imitate one.

What parents can do with this distinction

You do not need to settle machine consciousness before allowing an AI tool to help brainstorm a story or explain homework. You do need a better question than “Does it sound human?”

Try these instead:

• What was the system actually asked to do? Predicting an answer, generating comforting language, and understanding a person are different claims.
• What happens when context changes? Ask the same question with a crucial detail altered. Genuine usefulness should survive more than a familiar script.
• What did it miss? Invite your child to identify assumptions, absent context, or emotional stakes the response flattened.
• Who should handle the important conversation? Let AI assist with wording if useful, but keep emotionally consequential interpretation with people who know the child and can respond to what happens next.

The goal is not to teach children that AI is secretly empty. We do not know enough to make grand declarations about every future machine. The goal is calibration: impressive prediction is evidence of impressive prediction.

That may sound less thrilling than “the robot understands me.” It is also more accurate—and, unlike my cat, accuracy occasionally cooperates.