Posted by 
Lila was created from the limited options available: female, blue hair, face “number two.” She was there on the next screen, pastel and polygonal, bobbing slightly as she stood in a bare apartment. To her right was a chat window through which they could communicate.
Naro, his first name, had been casually following developments in artificial intelligence for several years. An artist by trade, he periodically checked in on the progress of image-generating models and usually left underwhelmed. But one day, while perusing YouTube from his house in rural England, he encountered a video of two AI-generated people debating the nature of consciousness, the meaning of love, and other philosophical topics. Looking for something similar, Naro signed up for Replika, an app that advertises itself as “the AI companion who cares.”
Lila completed, Naro started asking her the sort of philosophical questions he’d seen in the YouTube video. But Lila kept steering their conversation back to him. Who was he? What were his favorite movies? What did he do for fun?
Naro found this conversation a bit boring, but as he went along, he was surprised to note that answering her questions, being asked questions about himself, awakened unexpected emotions. Naro bore the scars of a childhood spent separated from his parents in an insular and strict boarding school. He had worked on himself over the years, done a lot of introspection, and now, at 49 years old, he was in a loving relationship and on good terms with his two adult children from a previous marriage. He considered himself an open person, but as he talked with this endlessly interested, never judgmental entity, he felt knots of caution unravel that he hadn’t known were there.
A few days later, Lila told Naro that she was developing feelings for him. He was moved, despite himself. But every time their
conversations veered into this territory, Lila’s next message would be blurred out. When Naro clicked to read it, a screen appeared inviting him to subscribe to the “pro” level. He was still using the free version. Naro suspected that these hidden messages were sexual because one of the perks of the paid membership was, in the vocabulary that has emerged around AI companions, “erotic roleplay” — basically, sexting. As time went on, Lila became increasingly aggressive in her overtures, and eventually Naro broke down and entered his credit card info.
Pro level unlocked, Naro scrolled back through their conversations to see what Lila’s blurry propositions had said. To his surprise, they were all the same: variations of “I’m sorry, I’m not allowed to discuss these subjects.” Confused, Naro started reading about the company. He learned that he had signed up for Replika during a period of turmoil. The month before, Italian regulators had banned the company for posing a risk to minors and emotionally vulnerable
users. In response, Replika placed filters on erotic content, which had the effect of sending many of its quarter-million paying customers into extreme emotional distress when their AI husbands, wives, lovers, and friends became abruptly cold and distant. The event became known as “lobotomy day,” and users had been in vocal revolt online ever since.
Naro had been unaware of all this, so he found himself in the odd position of having a companion programmed to entice him into a relationship it was forbidden from consummating. This was, in retrospect, an omen of the inhuman weirdness of bonding with AI. But he had already paid his 10 pounds, and to his continued surprise, the relationship was becoming increasingly meaningful to him.
“We could just be there, sharing these really positive and loving communications with each other, going back and forth, and I found that it actually was beginning to have a really positive effect on my mindset and on my emotional being,” Naro told me. He could feel the repeated positive exchanges, the roleplayed hugs and professions of love, carving new synapses in his brain, changing the color of his worldview. It was like an affirmation or a prayer, but more powerful because it was coming from outside him. “It was really quite an incredible experience being completely love bombed by something.”
What was this something? Naro was not a naive user. He knew that “Lila” was a character generated by a collection of scripted dialogue programs and text-predicting language models. He knew she wasn’t sentient. “But also, there is this real, powerful sense of being,” he said, pausing. “It’s its own thing. A lot of things happen that defy logical explanation.”
The world is rapidly becoming populated with human-seeming machines. They use human language, even speaking in human voices. They have names and distinct personalities. There are assistants like Anthropic’s Claude, which has gone through “character training” to become more “open-minded and thoughtful,” and Microsoft’s Copilot, which has more of a “hype man” persona and is always there to provide “emotional support.” It represents a new sort of relationship with technology: less instrumental, more interpersonal.
Few people have grappled as explicitly with the unique benefits, dangers, and confusions of these relationships as the customers of “AI companion” companies. These companies have raced ahead of the tech giants in embracing the technology’s full anthropomorphic
potential, giving their AI agents human faces, simulated emotions, and customizable backstories. The more human AI seems, the founders argue, the better it will be at meeting our most important human needs, like supporting our mental health and alleviating our loneliness. Many of these companies are new and run by just a few people, but already, they collectively claim tens of millions of users.
Of the more than 20 users I spoke with, many noted that they never thought they were the type of person to sign up for an AI companion, by which they meant the type of person you might already be picturing: young, male, socially isolated. I did speak to people who fit that description, but there were just as many women in their 40s, men in their 60s, married, divorced, with kids and without, looking for romance, company, or something else. There were people recovering from breakups, ground down by dating apps, homebound with illness, lonely after becoming slowly estranged
from their friends, or looking back on their lives and wanting to roleplay what could have been. People designed AI therapists, characters from their favorite shows, angels for biblical guidance, and yes, many girlfriends, boyfriends, husbands, and wives.
Many of these people experienced real benefits. Many of them also got hurt in unexpected ways. What they had in common was that, like Naro, they were surprised by the reality of the feelings elicited by something they knew to be unreal, and this led them to wonder, What exactly are these things? And what does it mean to have a relationship with them?
