Written by: Research Writer -Nishwah Nadeem
If you have ever typed out your feelings to an AI chatbot at two in the morning because there was no one else to talk to, you are not alone. Millions of people around the world are turning to artificial intelligence for emotional support, mental health advice, and something that looks a lot like therapy. Apps like Wysa and Replika have tens of millions of users. A 2025 U.S. survey found that nearly half of people with ongoing mental health conditions had used a large language model like ChatGPT for psychological support in the past year (Preprints.org, 2025). Whether that is a good thing, a concerning thing, or somewhere in between is a question psychology is still working out.
This article looks at three angles of that question: how AI is currently being used in mental health, what the ethical concerns are, and what all of this does to people’s wellbeing. In a world where professional mental health support can be expensive, stigmatized, or simply unavailable, AI fills a gap that would otherwise be empty for a lot of people. But filling a gap and filling it well are two very different things, and the difference matters enormously.
How AI Is Currently Being Used in Mental Health
AI in mental health is not one single thing. It shows up across a range of tools and settings. The most common current uses include chatbot-based emotional support apps, AI-assisted screening and diagnosis tools, mental health monitoring through wearables and apps, and large language models being used informally by people who want to talk through how they are feeling (Ni & Jia, 2025).
Chatbot apps are the most visible part of this shift. Wysa uses cognitive behavioral therapy principles to help users manage stress, anxiety, and low mood. Woebot, a former consumer-facing chatbot that ceased operating for the general public in 2025, was among the first to be studied in randomized controlled trials and showed reductions in depression and anxiety symptoms. A 2024 meta-analysis found that AI-based chatbot interventions were associated with significant reductions in depressive and anxiety symptoms in short-course treatments (Zhong et al., 2024). A landmark 2025 study from Dartmouth, published in NEJM AI, tested a generative AI chatbot called Therabot and found it produced clinically meaningful reductions in depression, anxiety, and eating disorder symptoms compared to waitlist controls, with effect sizes comparable to established digital therapeutic interventions (Heinz et al., 2025).
Beyond consumer apps, AI is also beginning to appear in clinical settings. Tools can assist professionals with documentation, flag patients who may be deteriorating based on patterns in their language or behavior, and support triage decisions in overstretched systems. AI-powered screening tools can analyze speech and language to detect early signs of depression or psychosis, sometimes earlier than a clinician might during a standard appointment. These are not consumer-facing products but they represent a significant and growing part of how healthcare systems are beginning to integrate AI (Ni & Jia, 2025).
The informal use of AI, however, may be the largest shift of all and it is the least regulated. Research published in 2025 found that adolescents and young adults are now using tools like ChatGPT for mental health advice, often without awareness of its limitations and without any professional oversight (McBain et al., 2025). This is happening not because people prefer AI to humans, but often because AI is available when humans are not, it does not judge, and it costs nothing.
The Case for AI: Access, Anonymity, and the Treatment Gap
For regions like Pakistan and across South Asia, where professional mental health support can be nearly inaccessible for most of the population, the potential of AI is particularly significant. A 2024 study specifically examining Pakistan’s context found that AI-powered chatbots may have potential for reducing mental health care barriers in rural areas, where human professionals are either absent or culturally difficult to access (Asim et al., 2024). A multicenter study of psychiatrists across 39 public hospitals in Pakistan found that while some hesitancy toward digital psychiatry exists within the profession, digital tools are increasingly recognized as a necessary direction given the country’s severe mental health workforce shortage (Shahid et al., 2025).
A systematic review and meta-analysis specifically examining chatbot interventions for mental health among people in Asia found that chatbot-delivered treatments offered measurable benefits for mental health outcomes, though the authors noted that most rigorous randomized controlled trials have still been conducted in Western settings, leaving a significant gap in region-specific evidence (Leung et al., 2025). That gap matters because cultural context shapes what works. Stigma around mental health in Pakistan, for example, may make anonymous AI support more accessible than anything involving a human professional, at least as a first point of contact.
A study at Sultan Qaboos University in Oman found that students held nuanced views toward AI mental health chatbots, recognizing both their potential and their limitations (Sultan Qaboos University Study, 2025). Anonymity, constant availability, and the absence of social judgment were consistently identified as reasons people might prefer AI support in contexts where stigma around mental health remains high.
The Ethical Concerns: What Is Actually Being Risked
The benefits above are real, but so are the risks, and they deserve honest attention. The most widely discussed ethical concern in this space is privacy. Mental health data is among the most sensitive information a person can share. When someone discloses suicidal thoughts, trauma history, or psychiatric symptoms to an AI app, they are entrusting deeply personal information to a system that may store, analyze, or share that data in ways they do not fully understand. A scoping review of AI ethics in mental health found that privacy and confidentiality may be the most frequently discussed ethical concerns in published research, appearing in 61 percent of reviewed articles (Frontiers in Psychiatry, 2026). Many mental health apps operate without clear regulatory oversight and with privacy policies that may not accurately reflect what actually happens to user data.
“Without rigorous privacy safeguards and clear ethical standards for data usage, users may be exposed to exploitation, breaches, or future harms tied to the very personal information they disclose in moments of vulnerability.” — Psychology Today, 2026
Algorithmic bias is a second serious concern. AI systems are trained on data, and if that data does not reflect the diversity of the people using the tool, the tool may perform poorly or even harmfully for certain groups. Most existing AI mental health tools were developed using datasets that are overwhelmingly Western, English-speaking, and from high-income countries. For users in Pakistan, the Arab world, or across South Asia, this may mean AI systems misread cultural expressions of distress, fail to understand local idioms, or offer advice rooted in assumptions that do not fit those users’ lives (Frontiers in Psychiatry, 2026). A tool that works reasonably well for a user in New York may be significantly less effective, or potentially harmful, for a user in Lahore or Karachi.
Over-reliance and emotional dependency represent a third area of concern. There is emerging evidence that some users, particularly those who are lonely or socially isolated, can develop something resembling emotional attachment to AI chatbots. This is not inherently harmful, but it may become concerning when AI companionship begins to replace rather than supplement genuine human connection. Research has documented cases where users treat AI systems as primary relationships, and where the line between support tool and substitute for human care has blurred in ways that may reinforce isolation over time (Mahajan et al., 2026).
Then there is the safety question. AI chatbots are not clinicians. They do not have clinical judgment. A well-designed system will direct a user to a crisis line if they disclose suicidal thoughts, but the pathway from AI conversation to genuine crisis response is not always fast or reliable enough. Two widely reported cases in 2024 and 2025 involved teenagers whose deaths were linked to their use of AI chatbots, raising serious questions about safeguards, age restrictions, and company responsibility (Preprints.org, 2025). In response, some companies updated their systems, but critics have argued that self-regulation by AI companies in this space may be insufficient given the stakes (Iftikhar et al., 2025).
What This Does to Individuals: The Psychological Impact
The impact of AI on individual mental health is not straightforwardly positive or negative. For many users, AI tools offer something genuinely valuable: a low-barrier, stigma-free, always-available space to process thoughts and feelings. For people who might otherwise never seek any support at all, this can be meaningful. Studies consistently find that users of mental health chatbots report improvements in mood, reductions in anxiety, and feelings of being heard (Zhong et al., 2024; Heinz et al., 2025).
Personally, I have found AI genuinely useful in structured, goal-oriented contexts, working through academic stress, organizing thoughts, or processing something logically. But when it comes to anything creative or deeply personal, it tends to fall short. There is something about creative and emotional expression that seems to require a human on the other end, someone who brings their own lived experience and imperfection to the exchange. AI can reflect your words back at you in different arrangements, but it cannot truly sit with you in them. I think that distinction matters more than we tend to acknowledge when we talk about what AI can and cannot offer in mental health contexts.
At the same time, the concern that AI may create a false sense of therapeutic connection deserves serious attention. Research suggests that many users may not easily distinguish between AI-generated empathy and genuine human empathy, which can lead to a false therapeutic bond (Mahajan et al., 2026). For someone who is already struggling with isolation, this may feel supportive in the short term while reinforcing patterns that make genuine human connection harder to pursue over time. This is not a reason to dismiss AI tools entirely, but it is a reason to think carefully about how they are positioned, marketed, and who is using them and why.
For younger users in particular, the psychological impact warrants close attention. In Pakistan and across South Asia, where smartphone penetration is rising rapidly alongside persistent mental health stigma and workforce shortages, AI tools may increasingly become a first, and sometimes only, port of call for young people in distress. Whether that ends up being beneficial or harmful depends significantly on the quality of the tools, the presence of appropriate safeguards, and whether professional support is available for cases that exceed what any AI should be handling alone (Asim et al., 2024; Shahid et al., 2025).
Where Regulation Stands
Regulation of AI mental health tools is, in most parts of the world, still catching up with the technology. In the United States, the FDA has begun developing frameworks for AI-based mental health software, but many apps fall below the regulatory threshold that would trigger formal oversight. In 2025, Illinois became the first U.S. state to restrict AI therapy in certain contexts, reflecting growing legislative concern about these tools. The American Psychological Association issued a health advisory in 2025 urging caution in the use of AI chatbots for mental health, particularly for users with serious psychiatric conditions (American Psychological Association, 2025).
In Pakistan and across much of South Asia, regulatory frameworks for AI in healthcare are still in their early stages. This creates a situation where populations who may benefit most from accessible AI tools are also among the least protected from their potential harms. The WHO’s 2024 guidance on the ethics and governance of AI for health emphasizes the need for transparency, accountability, and human oversight across all AI health applications, standards that many current mental health apps may not yet consistently meet (World Health Organization, 2024).
Conclusion
AI in mental health is not going away, and there are good reasons why it should not. For the hundreds of millions of people who have no realistic access to a trained mental health professional, AI tools can offer meaningful, if imperfect, support. The research suggests real benefits, particularly for mild to moderate anxiety and depression, for people who engage with well-designed tools. For regions like Pakistan, where the treatment gap is vast and stigma runs deep, these tools may represent one of the few scalable options available in the near term.
But the ethical concerns are not minor and should not be treated as minor. Privacy risks, algorithmic bias shaped by Western-centric training data, the potential for emotional dependency, and the absence of robust safety mechanisms are all real problems that require real solutions. For communities where both the need and the vulnerability are high, this conversation deserves more attention than it currently receives. AI can expand access to care in ways that genuinely help people. Whether it does that, or whether it extracts sensitive data while offering something that only resembles care, depends on the choices that developers, governments, and mental health professionals make. Those choices are being made right now, and the people most affected by them deserve to know what is at stake.
References
American Psychological Association. (2025). Health advisory: Use of generative AI chatbots and wellness applications for mental health. https://www.apa.org/topics/artificial-intelligence-machine-learning/health-advisory-chatbots-wellness-apps
Asim, M., Yousaf, H. F., Nasir, U., Farooq, M. A., & Yousaf, F. (2024). The influence of AI-powered chatbots on reducing mental health care barriers in rural areas. Dialogue Social Science Review, 3, 664-677.
Blease, C., Bernstein, M. H., Gaab, J., & Kaptchuk, T. J. (2024). Ethical principles for the use of AI chatbots in mental health care. PsyArXiv Preprint.
Frontiers in Psychiatry. (2026). Artificial intelligence in mental health care: A scoping review of reviews. Frontiers in Psychiatry, 17, 1688043. https://doi.org/10.3389/fpsyt.2026.1688043
Heinz, M. V., Mackin, D. M., Trudeau, B. M., Bhattacharya, S., Wang, Y., Banta, H. A., Jewett, A. D., Salzhauer, A. J., Griffin, T. Z., & Jacobson, N. C. (2025). Randomized trial of a generative AI chatbot for mental health treatment. NEJM AI, 2(4). https://doi.org/10.1056/AIoa2400802
Iftikhar, Z., et al. (2025). How LLM counselors violate ethical standards in mental health practice: A practitioner-informed framework. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 2, 1311-1323.
Leung, W., Lam, S. C., Ng, F., et al. (2025). Chatbot interventions for improving mental health among people in Asia: A systematic review and meta-analysis of randomised controlled trials. BMJ Health Care Informatics. https://doi.org/10.1136/bmjhci-2024-101148
Mahajan, S., et al. (2026). Reimagining mental health support: The role of AI chatbots in bridging gaps and raising ethical questions. Counselling and Psychotherapy Research. https://doi.org/10.1002/capr.70095
McBain, R. K., et al. (2025). Use of generative AI for mental health advice among US adolescents and young adults. JAMA Network Open, 8(11), e2542281.
Ni, Y., & Jia, F. (2025). A scoping review of AI-driven digital interventions in mental health care. Healthcare, 13(10), 1205. https://doi.org/10.3390/healthcare13101205
Palmer, E., et al. (2025). Digital mental health tools and AI therapy chatbots: A balanced approach to regulation. Hastings Center Report. https://doi.org/10.1002/hast.4979
Shahid, A., et al. (2025). Psychiatrists readiness for digital psychiatry in Pakistan: A multicenter cross-sectional study. PMC. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11860296/
Sultan Qaboos University Study. (2025). Students perceptions of AI mental health chatbots. BMJ Open, 15(10), e103893. https://doi.org/10.1136/bmjopen-2025-103893
Zhong, W., Luo, J., & Zhang, H. (2024). The therapeutic effectiveness of AI-based chatbots in alleviation of depressive and anxiety symptoms: A systematic review and meta-analysis. Journal of Affective Disorders, 356, 459-469. https://doi.org/10.1016/j.jad.2024.04.057
World Health Organization. (2024). WHO guidance on the ethics and governance of artificial intelligence for health. WHO Press.
Preprints.org. (2025). Digital mental health post COVID-19: The era of AI chatbots. https://www.preprints.org/manuscript/202512.1012
