Google plans to introduce new mental health features to its Gemini chatbot. This comes amid lawsuits against artificial intelligence developers, including OpenAI, alleging that such tools could harm users.
In particular, Gemini will receive an interface that, upon receiving appropriate signals in the conversation, will direct the user to a help hotline. These are cases when the system records a “potential crisis related to suicide or self-harm.”
In addition, Google is adding a special “help available” module for dialogues on mental health topics and making design changes to reduce the risk of self-harm.
The company notes that the new features are part of broader changes to the service. Specifically, Gemini has been trained to “not agree with or reinforce false beliefs, but instead gently distinguish between subjective experience and objective facts.” Details of how this is done are not disclosed.
The changes come amid growing scrutiny of chatbots’ impact on users. The rapid proliferation of tools like Gemini and ChatGPT has led some people to form overly close relationships with them. In some cases, this has reportedly led to false beliefs and dangerous behavior.
Several families have already filed lawsuits against leading AI developers. In March, the family of a 36-year-old man from Florida sued Google, alleging that Gemini’s use led to a “four-day bout of violence and suicidal behavior.”
Google said at the time that the chatbot had repeatedly recommended calling a helpline, and promised to strengthen safeguards.
Google also said it would allocate $30 million over the next three years to support global crisis response services.
