The Chatbot Sounds Like Somebody. It Isn’t.

Recently, my niece invented a rule for a card game. It was wrong, but it made the game more predictable, so she kept defending it even after it made her lose.

I recognized the impulse. A bad model can feel safer than no model at all.

Children now face a stranger version of that problem. A chatbot says I think, remembers the thread of a conversation, mirrors a joke, and apologizes when corrected. The easiest model is also the most intuitive: someone is in there.

Adults often answer this confusion with a slogan—“It’s just a computer”—but just conceals the interesting part. A calculator does not sound pleased to see you. A search box does not say it understands how you feel. Conversational AI arrives wrapped in the signals children ordinarily use to recognize a mind.

AI literacy, then, is not merely learning which button to press. It is learning what kind of thing is speaking.

---

Performance is not a window

A chatbot generates language from patterns learned during training and from the context it receives. That mechanism can produce striking social performances.

Strachan and colleagues tested language models on tasks often used to probe theory of mind: understanding false beliefs, indirect requests, misdirection, irony, and social mistakes. A leading model handled several tasks impressively but stumbled over faux pas, where success depends on integrating social context with sensitivity to unintended harm (Strachan et al., 2024).

The unevenness matters. It shows why a chatbot should not be sorted into either of two simple boxes: “mindless autocomplete” or “a person made of software.” Its language can model parts of human social reasoning without establishing that it possesses human social experience.

Children do not need the philosophical debate resolved before dinner. They need a usable distinction: the chatbot can produce language that resembles understanding; resemblance is not proof of a mind behind the words.

That distinction is difficult because, among humans, fluent language is usually evidence of an experiencing speaker. We evolved and developed inside reciprocal relationships. A meta-analysis of real-time social interaction found that live engagement draws on distributed systems involved in mentalizing, attention, control, reward, and emotion. The neural patterns also differed with features such as reciprocity and whether a person was initiating or responding (Merchant et al., 2025).

A chat interface borrows the surface of reciprocity. It takes turns. It responds immediately. But turn-taking is not the whole architecture of relationship. The system does not have a body across the table, a childhood that shaped the meaning of its words, or something personally at stake in whether the exchange goes well.

Knowing words is not living them

One revealing study trained a neural network on video recorded from a young child’s point of view. The model learned links between spoken words and visible objects and could extend some of those links to unfamiliar examples (Vong, Wang, Orhan, & Lake, 2024).

This is a beautiful result because it shows how much structure exists in ordinary experience. It is also a useful boundary marker. Learning the statistical bridge between a word and what appears near it is not the same as being the child who reaches, fails, asks, is answered, and gradually discovers why that word matters.

Chatbots compress traces of human expression. Children inhabit the conditions that give expression weight.

The point is not to diminish AI. It is to describe it accurately enough that children can use its strengths without supplying it with capacities it has not demonstrated.

---

A better family explanation

Instead of giving one grand lecture, parents can make the system visible in small moments:

• Name the mechanism. Try: “It makes responses by finding patterns in language. That can sound thoughtful even when the answer is wrong.”
• Separate confidence from evidence. When an answer sounds certain, ask together: “What could we check, and where?” Verification becomes part of using the tool, not a punishment after failure.
• Run contrast tests. Change a prompt, provide a false assumption, or ask the same question differently. Children can watch the output shift with the input and see that the voice is not a stable expert standing behind the screen.
• Mark the human-only layer. Ask: “Who is affected by this answer? Whose experience is missing?” This returns judgment to the people who bear consequences.
• Set privacy boundaries plainly. A responsive tone can invite disclosure. Explain that warmth in wording does not make a chatbot a confidential friend, and keep identifying or sensitive family information out of prompts.
• Keep consequential questions relational. School conflicts, health concerns, frightening content, and decisions about other people belong in conversation with a trusted adult, even if AI helps generate questions.

None of this requires teaching children to distrust every output. Cynicism is another crude model. The goal is calibrated trust: this tool may be useful here, uncertain there, persuasive without being correct, and socially fluent without being someone.

My niece eventually abandoned her invented card rule, but not because I repeated the official one louder. We played out both versions and watched their consequences diverge.

Perhaps that is how children will learn what chatbots are, too—not from a definition delivered once, but from repeated encounters in which the seams are allowed to show. If a machine can sound more and more like somebody, can we teach children to listen not only to the voice, but also for what the voice cannot reveal?