.png)
6 hours ago
5
Do you ever get the feeling that the people running the world are delulu? That the 1% are living in a completely different universe from the rest of us? You’re not the only one. Even some tech elites are starting to worry about their peers’ grasp on reality. “CEOs are uniquely prone to AI psychosis,” Aaron Levie, a co-founder of the enterprise cloud company Box, declared on X last month. His reasoning for this? “They’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI. So when they play with AI, they see the happy path results, often not considering the next 10 or 20 things that have to happen to get sustainable results from agents.”
In other words: CEOs are so high up the food chain that they don’t understand the human labour that goes into turning an error-riddled AI creation into something that functions properly in a business context. They are desperate to replace their annoying and expensive human labour with compliant AI models, but grossly overestimate what the technology can do. Meanwhile, the industry is rushing out overhyped AI solutions without properly stress-testing them.
This collective euphoria has resulted in some predictable disasters. In April, an AI coding agent powered by Anthropic’s Claude went berserk and deleted a company called PocketOS’s entire production database, along with backups. Jeremy Crane, PocketOS’s founder, later mused on X that this sort of failure was “inevitable” because the industry is “building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe”. To recall Facebook’s old mantra: it’s moving fast and breaking things.
Those things include our brains and grasp on reality. There’s a viral quote floating around the internet that quips: “The dumbest person you know is currently being told ‘You’re absolutely right!’ by ChatGPT.” Our tech overlords designed AI chatbots to be obsequious because it’s good business: having your opinions and feelings constantly validated increases user engagement.
But what else does this constant flattery do? While we’re still figuring that out, early studies aren’t reassuring. Research published in the Lancet Psychiatry in March found chatbots can encourage delusional thinking, particularly in people already vulnerable to developing psychotic symptoms. And a recent study from Stanford computer scientists found LLM “sycophancy can undermine users’ capacity for self-correction and responsible decision-making”. There is a pressing need, the study stresses, “to address AI sycophancy as a societal risk”.
Forget AI for a second; it’s already clear sycophancy is a societal risk. Just look at Donald Trump: his entire administration functions like a fawning chatbot. With every inane utterance that comes out of his mouth, a chorus of minions chirp: “You’re absolutely right!” Nobody close to him has the courage to rein him in. The same was true to some degree of Joe Biden. We might not be where we are now if his inner circle had provided tough love and told him that he shouldn’t pursue a second term. Instead, numerous political insiders have said that Biden surrounded himself with a tight group of “yes men” who shielded him from negative criticism. Meanwhile, in a corporate context, various studies have found a strong correlation between incessant flattery and poor executive performance. The powerful have always been able to construct their own realities; now, however, we’ve got machines that do it on an industrial scale.
Levie’s comments about AI psychosis, which have sparked intense debate, were really about CEOs having unrealistic expectations of a technology they can’t stop talking about but don’t fully understand, rather than true psychosis. But I think Levie could have gone much further in his diagnosis. In certain circles, there is an almost religious reverence for AI (I’ve previously called it AI-theism). This LLM-worship is causing some of the most powerful people in the world to direct their energy and resources towards building a “transhuman” future, where we (or, perhaps more accurately, a privileged few) merge with machines. A small but growing group of technologists are even yearning for a posthuman future where AI effectively replaces us. Guardrails be damned: they want to accelerate at full speed towards that future – speeding up the climate crisis with resource-hungry datacentres – even if it means the end of the world as we know it.
What a stupid, but perhaps fitting, way for humanity to end. An asteroid took the dinosaurs out. Now it’s looking increasingly likely we’re going to be wiped out because a cohort of rich losers keep being told “You’re absolutely right!” by ChatGPT.
Arwa Mahdawi is a Guardian columnist
