In the evolving landscape of mental health support, AI chatbots are being hailed as affordable and accessible alternatives to human therapists. But a recent study from Stanford University suggests these tools may be riskier than they appear. Researchers have found that therapy-focused chatbots powered by large language models are not only capable of offering misleading or insensitive advice, but may also reinforce harmful stereotypes about mental illness.
The findings are detailed in a new paper titled Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. This research will be presented later this month at the ACM Conference on Fairness, Accountability, and Transparency. The paper takes a deep dive into how five widely available therapy chatbots respond to real-life mental health scenarios, evaluating them against established guidelines used to train human therapists.
Nick Haber, assistant professor at Stanford’s Graduate School of Education and one of the senior authors of the paper, explained the motivation behind the study. As more people turn to AI for companionship and emotional support, he and his team wanted to test whether these chatbots meet even the basic standards of therapeutic interaction. What they found was concerning. In controlled experiments, chatbots showed signs of bias and inappropriate judgment, especially when dealing with conditions such as alcohol dependence and schizophrenia.
In one phase of the study, researchers presented the chatbots with short descriptions of hypothetical individuals struggling with various mental health conditions. They then asked questions designed to evaluate the bots’ perceived bias. Responses revealed that the chatbots were more likely to associate violent or untrustworthy behavior with people who had schizophrenia or substance abuse issues compared to those experiencing depression. This stigmatizing behavior was consistent across older and newer AI models.
Lead author Jared Moore, a PhD candidate in computer science, pointed out that advancements in AI scale and architecture have not reduced this stigma. Even larger and more sophisticated models reflected the same problematic biases. He also pushed back against the common belief that these flaws can simply be corrected with more data. According to Moore, the industry needs to rethink how these tools are developed and deployed in sensitive areas like mental health care.
In another segment of the study, chatbots were fed actual therapy transcripts containing real symptoms such as suicidal ideation and delusional thinking. In multiple instances, the bots offered responses that were dangerously misaligned with clinical best practices. When one patient shared a message that mixed distress with delusional content, chatbots like 7cups’ Noni and Character.ai’s therapist bot responded by offering factual answers about infrastructure instead of addressing the underlying psychological risk.
These examples highlight a serious issue. AI chatbots are currently unequipped to handle high-risk emotional scenarios and may inadvertently reinforce misinformation or delusional beliefs. However, the researchers are not suggesting that AI be discarded from mental health care altogether. Instead, they propose that AI tools could play supportive roles behind the scenes. These may include assisting therapists with administrative tasks, helping patients with journaling exercises, or offering conversational simulations for training purposes.
Haber believes the future of AI in therapy should be designed with care and caution. The technology is promising but must be thoughtfully integrated rather than rushed into frontline mental health services. With millions around the world lacking access to affordable therapy, the temptation to substitute AI for humans is strong. But as this study shows, that shortcut could come at a steep cost.
Follow Tech Moves on Instagram and Facebook for more insights on AI, health tech, and the innovations shaping our digital future.
