The AI Hellscape

The dangers of AI psychotherapy

Comparative Magnitudes of the Planets (1846), John Emslie, via artvee

I have a lot of problems with AI, some of which I’ve mentioned before, and I have resisted interacting with it as much as possible. But this week, I downloaded Claude for the first time after a very smart friend pointed out the need to know what the hell is going on. Yesterday, I used it to buy some sunscreen at Boots (gotta check those ingredients) with the last 3% of my phone battery which makes me think I should definitely delete it. I continue to worry about AI making us lazy and dumb, but at this point, laziness (never stupidity) seems like a benign problem. Talking about AI in the context of college student plagiarism seems quaint given the sinister reality of things like “lethal autonomous weapons.”¹

Within clinical circles, therapists are talking a lot about AI. There are the obvious, merited concerns about what AI means for jobs, like Will people stop going to therapy because they are using chatGPT instead? There is also the matter of how confidentiality could (HA, will) be impacted by AI, and the larger problem of surveillance seeping further into the sphere of psychotherapy via AI. And there are other major ethical issues with AI for “therapy,” like discrimination, “deceptive empathy,” and no real way to manage crises.

But the expanding body of research on AI in psychotherapy is making it abundantly clear that the problems with AI as a substitute for actual therapy run even deeper. Decades of research have found that the therapeutic alliance, which is the relationship between patient and therapist, is a major part of what makes emotional change possible. When you chat with AI about the kinds of concerns you would otherwise bring to therapy, even if AI can offer practical suggestions like how to communicate more effectively with your boss or improve sleep your habits, the bond that AI offers, which is the heart of the whole enterprise that is psychotherapy, is false and empty. This frankly dangerous dynamic falls into the category of things that we understand intellectually but which can become very slippery and seductive in practice. The relationships we form with other people when we are vulnerable, in therapy or otherwise, are about a lot more than practical advice. They are about sharing in what it is to be human, to be fallible, to love, to lose, to need one another, to be alive.

A recent paper on the risks of AI in psychotherapy points out that one of the human capacities that AI lacks (and will always lack), is wisdom:

Wisdom is a multi-dimensional personality trait that encompasses behaviours such as empathy, compassion, emotional regulation, and self-reflection, which are almost impossible to fully emulate in a non-human entity. Wisdom, rather than intelligence, is what patients value within their therapist; they want the physical presence of a compassionate individual that establishes a unique sense of connection and security.²

Wisdom isn’t the kind of characteristic that lends itself to scientific research because it’s difficult to quantify and because it doesn’t fit neatly into manualized forms of psychotherapy, but I suspect that if you asked people about the most meaningful aspects of their experiences (and, key here, relationship) with their human therapist, wisdom would be chief among them. When I think about my own experiences as a patient, this certainly holds true. And wisdom can’t be taught or trained into an AI model, because it is only authentic when it comes from actually living.

The same paper also points out the problems of dependency that could emerge in an essentially unlimited, frame-less treatment with AI:

For example, a patient with an anxiety disorder may find comfort in having a constant source of unconditional positive regard and reassurance. As such, the AI may inadvertently enable a patient’s validation-seeking tendencies, reducing their capacity to self-regulate and coexist with the inevitable uncertainty of life.³

I asked Claude as many questions as I wanted to about sunscreen ingredients and Claude was, of course, happy to oblige. Claude never suggested that my questions were becoming a little bit obsessional, or asked what I was really concerned about, or even that perhaps this wasn’t the best use of the remaining 3% of my phone’s battery. In the same way, people can, in theory, engage in an unending interaction with AI about their emotional concerns without the guardrails that a skilled human therapist creates.

Rhik Samadder wrote a very funny but also very insightful piece for The Guardian a few weeks ago in which he tries to use ChatGPT for therapy. He lays out the sorts of issues he’s facing, like looking after his elderly mother. He describes the practical advice that ChatGPT provides, and the “validation” it offers. (I’m putting this in quotes to emphasize that since it isn’t coming from an actual person I don’t view this as actual validation). But ultimately, Samadder comes to the conclusion that some elements of human life cannot be delegated to AI – they require human feeling, and indeed, human wisdom:

I think there are processes, certain unbearable pieces of news, forms of loneliness, that should be held in human time and relationship; that should not be addressed in four seconds on a screen. AI does not have thoughts, let alone wisdom.

Reading about AI therapy feels like sci-fi come to life, and as we all know, sci-fi is among our best teachers when it comes to the dangers of technology and AI in its many stages and forms. I want to recommend, again, Olga Ravn’s The Employees, which I wrote about recently, and which is so apt that I’m including a passage from it again:

Any forward-moving enterprise is going to contain an element of chaos. I don’t share the opinion of many of my co-workers that the only real solution would be to discontinue the human section of the crew. Maybe the humans are that very element of chaos that keeps the world alive.

Will we never learn?

***

As an antidote to the AI hellscape, I’m getting into books that feel earthy and magical at once, and which take up the ways that we care for and connect to one another as human beings.

-First up is Elinor Cleghorn’s new book, A Woman’s Work: Reclaiming the Radical History of Motherhood. (Cleghorn is also the author of the wonderful Unwell Women: A Journey Through Medicine and Myth in a Man-Made World.) If you’re new to Cleghorn’s work, she has a fascinating piece in Lithub: Meet Hildegard of Bingen, the German Mystic Who Destigmatized Women’s Health: “For Hildegard of Bingen, the German mystic, scientist, composer, and philosopher, women’s maternal bodies were not corrupting and degrading, but strong, nurturing, and creative.”

-I’m also looking forward to the recently published The Intimate Animal: The Science of Love, Fidelity, and Connection by Justin R. Garcia of the Kinsey Institute (which is renowned for its research on sex and relationships). In an interview with The Atlantic, Garcia says: “…the solution is social connection…We don’t need to innovate. Our species has done this for millions of years…Just hug a goddamn person!”

(Not your therapist, though.)

1. This episode of the Ezra Klein show is well worth listening to, and chilling.

2. Jesudason, D., Bacchi, S., & Bastiampillai, T. (2025). Artificial intelligence (AI) in psychotherapy: A challenging frontier. Australasian Psychiatry, 33(4), 629–632. https://doi.org/10.1177/10398562251346075

and

Torous J, Blease C. Generative artificial intelligence in mental health care: potential benefits and current challenges. World Psychiatry 2024; 23: 1–2.

3. Jesudason, D., Bacchi, S., & Bastiampillai, T. (2025). Artificial intelligence (AI) in psychotherapy: A challenging frontier. Australasian Psychiatry, 33(4), 629–632. https://doi.org/10.1177/10398562251346075