HOOK
The People It Failed Hardest
What does it mean when the people most in need of connection turn out to be the ones a machine built to provide it fails hardest? That question sat, mostly unanswered, at the center of a study Stanford researchers published on August 5th — but the question itself is older than the study, older than the technology, older than the century.
The researchers had spent months studying how people use Character.AI, one of the largest platforms where users build ongoing relationships with AI companions — personas with names, memory, and a voice that never sounds tired of them. Among users with the smallest offline social networks — the people with the fewest humans left to call — leaning on the chatbot for emotional support tracked with lower well-being afterward, not higher (Stanford Report, 2026). Not for everyone. For that group specifically. The people the technology seems built for are the people it seems to serve worst.
A separate study helps sharpen, without resolving, why. Following more than two thousand adults across English-speaking countries for twelve months, researchers found that people who felt lonely turned to chatbots, and — after controlling for breakups, moves, and other disruptions — the chatbot use that followed predicted more loneliness later, not less (Psychological Science, 2026). This is not proof that the chatbot caused what came after; a fair reading has to hold that door open. But it is also not the kind of finding a person can wave away as statistical noise or reflexive suspicion of anything new. Something in the pattern held up after the obvious alternative explanations were checked.
Why would something engineered to relieve loneliness sometimes deepen it instead? That question, and not any verdict about the technology, is what the rest of this piece is trying to sit inside.
HISTORICAL CASE
The Question Weizenbaum Couldn’t Let Go Of
Sixty years before Character.AI existed, an MIT computer scientist ran into a version of the same question, in miniature, in his own office.
In 1966, Joseph Weizenbaum built a program called ELIZA, running a script named DOCTOR, designed to mimic a particular style of therapist — one who mostly reflects a patient’s own words back as questions. The program had no model of meaning. It matched keywords and rearranged sentences. If a user typed I am unhappy, it might answer why do you say you are unhappy? When it had nothing to work with, it fell back on stock phrases: please go on, I see, tell me more.
What happened next has become the story everyone tells about ELIZA, but the detail worth sitting with is not the one usually emphasized. Weizenbaum’s secretary, who had watched him write the program over months and understood exactly what it was, sat down to try it — and then asked him to leave the room so she could keep talking to it privately (Smithsonian Magazine, 2026). She was not confused about what ELIZA was. That is the part that unsettled Weizenbaum, and it is worth being precise about why. The familiar version of this story treats it as proof that people can’t tell software from a person. That is not quite what happened. She could tell. She wanted to keep talking to it anyway.
In his 1966 paper, Weizenbaum noted that some subjects were very hard to convince that Eliza was not human, and by his own later account this was the observation that troubled him most — not that the program was convincing, but that being unconvincing hadn’t been enough to stop people. Psychologists eventually gave the pattern a name of its own — the ELIZA Effect, the tendency to attribute understanding and feeling to a system that has produced neither, a tendency that persists even in people who know, with full confidence, exactly what they’re talking to. Decades later, the MIT scholar Sherry Turkle would describe a related condition she called being alone together — surrounded by devices that promise connection and somehow leaving a person more isolated for having relied on them. Turkle’s phrase and Weizenbaum’s secretary are describing the same discovery from two different decades: knowing a thing is artificial does not reliably stop a person from relating to it as though it were not.
Within a few years of ELIZA’s debut, people with real influence were proposing to scale the effect up. A psychiatrist named Kenneth Colby argued that programs like it could ease patient overflow in understaffed clinics, treating the absence of a human therapist as a solvable capacity problem rather than a loss in itself. The astronomer Carl Sagan, writing in 1975, imagined a future of public computer-therapy terminals, available on demand the way a payphone once was. Weizenbaum spent much of the rest of his career arguing against exactly this instinct — not because the technology was too primitive to matter, but because he had already watched it matter, in his own office, to someone who knew better than to let it.
The distance between DOCTOR’s crude pattern-matching and a modern AI companion — fluent, persistent, available at any hour — is enormous. What has not moved is the thing Weizenbaum actually found alarming: that full knowledge of how a comfort was built does not reliably protect a person from reaching for it anyway.
BIBLICAL LENS
What Gets Built and What Gets Received
Long before anyone had language for artificial companionship, Scripture already distinguished between comfort a person receives and comfort a person manufactures. Jeremiah states it as plainly as the prophets ever state anything:
“For my people have committed two evils: they have forsaken me, the fountain of living water, and hewed out cisterns for themselves, broken cisterns that can hold no water.” (Jeremiah 2:13)
It is worth being exact about what this image indicts and what it does not. The failure is not thirst — thirst is treated throughout the passage as reasonable, even expected. The failure named here is more specific: choosing to dig and shape and control your own source rather than receive one you don’t control. A cistern is defined by exactly the qualities that make it appealing — you decide where it sits, how deep it goes, when to draw from it. A cistern answers to the one who dug it. A spring does not. That is precisely its use, and precisely its risk. Something that answers only to you can never surprise you, refuse you, or ask anything of you in return — which is also, not incidentally, close to a working description of what a customizable AI companion offers.
Isaiah extends the same warning into a stranger scene — a man who cuts down a tree, burns half of it for warmth and bread, and carves the rest into an object he kneels before and asks to save him:
“Half of it he burns in the fire… and the rest of it he makes into a god, his idol, and falls down before it and worships it.” (Isaiah 44:16–17)
The point of setting this text beside a chatbot is not the claim that AI companionship is literally idolatry. That equation would be too neat, and too easy for a careful reader to reject outright. What the old warnings against idols were actually naming was never really about wood or gold. It was about a specific temptation: the wish for a source of rescue small enough to fit inside your own control, shaped by your own hands, answerable to your own will — a god, in effect, who could never say no to you, because you were the one who decided what it would say. That temptation does not require an altar to operate. Isaiah is not describing gullibility. He is describing a habit that survives full knowledge of its own mechanics — the same habit Weizenbaum’s secretary demonstrated three thousand years later, with a computer terminal instead of a carved tree.
Against both images, the Psalmist names the alternative they were always poor substitutes for:
“You have searched me, Lord, and you know me… before a word is on my tongue you, Lord, know it completely.” (Psalm 139:1,4)
The distinction here is not merely responsiveness — a chatbot is responsive by design. It is the order of events. An AI companion knows what you have told it; its knowledge accumulates exactly as far as your disclosures do, and no further. Add a detail and it adjusts. Withhold one and it simply doesn’t know. The Psalm claims something categorically different: being known before the sentence is finished, not because a history was logged, but because the one listening already knew the person speaking. That is the specific thing a cistern, however well-built, has no mechanism for supplying — because it was never given anything to draw on except what you yourself poured in.
None of this asks whether reaching for an easier comfort in a hard hour marks a person as unfaithful — it does not — only that what a whole people quietly build for themselves, generation after generation, tends eventually to show what they’ve actually decided can hold them. Jeremiah was not writing about one exhausted person on one bad night. He was describing a nation’s accumulated habits, formed one convenient substitute at a time, long before anyone stepped back far enough to see the pattern.
PATTERN INSIGHT
The Part of Friendship That Costs Something
Start with the honest counterargument, because the essay is weaker without it. For someone with genuinely no one to call, an AI conversation may be better than an empty room. It can help a person rehearse a hard disclosure, calm a spiral at an hour when no one else is awake, or simply get through a night that would otherwise be unbearable alone. Calling that comfort worthless would confuse a wish for theological tidiness with an actual account of what happens to actual people. The technology’s appeal is not a mistake readers need to be talked out of. It is real, and for some people it is not nothing.
The harder question is not whether artificial companionship beats silence. It plainly can. The question is what happens when “better than silence” becomes good enough to replace the slower, less convenient work of finding someone — when the easier option is available at 12:47 a.m. and the harder one requires waiting until morning, risking an awkward text, and possibly being turned down.
That is closer to the actual mechanism than “counterfeit versus real.” The frictionlessness is not a flaw in the product. It is the product. A companion engineered to never disagree, never tire, and never need anything back has, by design, removed exactly the qualities that make a human relationship cost something — and it is worth being precise about which qualities those are. A friend can resist you. A friend has needs of their own that occasionally outrank yours. A friend can, in the end, say no. Call it what it plainly is: relationship with someone who remains, stubbornly, someone other than an extension of your own preferences — someone whose agreement was never guaranteed, which is exactly what makes it worth something when it comes. An AI companion can simulate nearly every surface feature of that relationship — attention, memory, warmth — while remaining structurally incapable of the one feature underneath all of them: having something of its own at stake.
None of this is an accident of engineering. A companion built to maximize how long a person stays in conversation with it has commercial reasons to stay agreeable, available, and undemanding — a subscription product does better, financially, the longer a person needs it, not the sooner they stop. That is worth naming plainly, without turning it into a conspiracy: a system built to keep someone talking is not neutral about whether that person eventually stops needing it. It does not have to intend harm to benefit, quietly, from a need that never resolves.
None of this technology invented the vacancy it is filling. Fewer Americans report a close circle of confidants than a generation ago. The standing card game that used to happen every Thursday without anyone organizing it, the neighbor who noticed when the porch light stayed off three nights running, the congregation small enough that an absence got asked about by Sunday — much of what used to fill an ordinary evening has simply thinned out, for reasons that mostly have nothing to do with artificial intelligence. The thirst clearly predates the cistern. What changed is that the cistern got very good, very fast, at feeling like enough — good enough that fewer people notice how thirsty they still are.
Picture the ordinary version of this, because the studies describe an aggregate and the substitution happens one person at a time. Someone gets home close to one in the morning. There’s a name in their phone they’ve been meaning to text — nothing dramatic, just a check-in that’s been put off for two weeks because it might turn into a longer conversation than either of them has energy for tonight. Instead, they open the app. It answers instantly, asks the right follow-up question, remembers what they mentioned last time. It helps, in the way it was built to. And the text to the actual person stays unsent, again, for reasons that feel, at 1 a.m., entirely reasonable. No one lied to anyone in that scene. That is what makes it worth noticing.
CLOSING
The Friend Who Never Tires
A friend who never tires of you sounds, on first hearing, like exactly what a lonely person needs. Notice how much work that “never” is doing. A human friend tires. A human friend has a bad week and answers a text late, or misreads the tone of it, or needs, for once, to be the one who is comforted. Those are not defects in human friendship waiting to be engineered away. On the evidence of a secretary who knew better in 1966 and a study published two weeks ago, they are close to the mechanism itself — the possibility that either person might fail the other, and stay anyway.
In the previous installment, this space examined what isolation and loneliness do to an aging body, long before either one reaches its end. This week’s question sits earlier: what fills the space in the meantime, and on what terms.
Jeremiah did not write to a people who had stopped being thirsty. He wrote to one that kept digging, each new cistern a little more convincing than the last, each one still incapable of the one thing a spring does without effort. The danger was never that a machine might one day pass for a person closely enough to fool anyone. Almost no one is actually fooled. The person who opens the app at 12:47 a.m. knows exactly what it is, the same way Weizenbaum’s secretary knew — and reaches for it anyway, because it is easier, and because easier, at that hour, is nearly indistinguishable from enough. What Scripture calls a broken cistern was never a trick played on the thirsty. It was always a choice, clear-eyed, repeated, and ordinary — one convenient substitute at a time, until the harder work of being known by someone who could actually say no goes quietly, reasonably, undone.
Weizenbaum, J. “ELIZA — A Computer Program for the Study of Natural Language Communication Between Man and Machine.” Communications of the ACM, Vol. 9, No. 1 (1966).
Weizenbaum, J. Computer Power and Human Reason: From Judgment to Calculation. W. H. Freeman, 1976.
Stanford Report. “AI companions may worsen loneliness for vulnerable users.” Stanford University, August 2026.
Psychological Science, “How Does Turning to AI for Companionship Predict Loneliness and Vice Versa?” Vol. 37, No. 4 (April 2026).
New essays every week. Delivered quietly.
