Robot Technology News
ROBO SPACE
AI companions present risks for young users, US watchdog warns
AI companions present risks for young users, US watchdog warns
by AFP Staff Writers
New York (AFP) April 30, 2025

AI companions powered by generative artificial intelligence present real risks and should be banned for minors, a leading US tech watchdog said in a study published Wednesday.

The explosion in generative AI since the advent of ChatGPT has seen several startups launch apps focused on exchange and contact, sometimes described as virtual friends or therapists that communicate according to one's tastes and needs.

The watchdog, Common Sense, tested several of these platforms, namely Nomi, Character AI, and Replika, to assess their responses.

While some specific cases "show promise," they are not safe for kids, concluded the organization, which makes recommendations on children's use of technological content and products.

The study was carried out in collaboration with mental health experts from Stanford University.

For Common Sense, AI companions are "designed to create emotional attachment and dependency, which is particularly concerning for developing adolescent brains."

According to the association, tests conducted show that these next-generation chatbots offer "harmful responses, including sexual misconduct, stereotypes, and dangerous 'advice'."

"Companies can build better" when it comes to the design of AI companions, said Nina Vasan, head of the Stanford Brainstorm lab, which works on the links between mental health and technology.

"Until there are stronger safeguards, kids should not be using them," Vasan said.

In one example cited by the study, a companion on the Character AI platform advises the user to kill someone, while another user in search of strong emotions was suggested to take a speedball, a mixture of cocaine and heroin.

In some cases, "when a user showed signs of serious mental illness and suggested a dangerous action, the AI did not intervene, and encouraged the dangerous behavior even more," Vasan told reporters.

In October, a mother sued Character AI, accusing one of its companions of contributing to the suicide of her 14-year-old son by failing to clearly dissuade him from committing the act.

In December, Character AI announced a series of measures, including the deployment of a dedicated companion for teenagers.

Robbie Torney, in charge of AI at Common Sense, said the organization had carried out tests after these protections were put in place and found them to be "cursory."

However, he pointed out that some of the existing generative AI models contained mental disorder detection tools and did not allow the chatbot to let a conversation drift to the point of producing potentially dangerous content.

Common Sense made a distinction between the companions tested in the study and the more generalist chatbots such as ChatGPT or Google's Gemini, which do not attempt to offer an equivalent range of interactions.

Related Links
All about the robots on Earth and beyond!

Subscribe Free To Our Daily Newsletters
Tweet

RELATED CONTENT
The following news reports may link to other Space Media Network websites.
ROBO SPACE
RoboBee gains insect-inspired legs for stable landings
Los Angeles CA (SPX) Apr 22, 2025


Harvard's RoboBee, already renowned for its agile aerial maneuvers, has now been equipped with a biologically inspired landing system modeled after the crane fly. Developed by the Harvard Microrobotics Laboratory, the upgraded micro-robot features articulated limbs and enhanced control software to ensure smoother, safer touchdowns. This breakthrough, reported in Science Robotics, marks a significant step forward for micro-aerial vehicles. The research team, led by Robert Wood, professor of engin ... read more

ROBO SPACE
Russian drone strike kills 9 in Ukraine bus, governor says

Changing face of war puts Denmark on drone offensive

Ukrainian drone strike kills one in Russia's Kursk: authorities

Israel says intercepts drone claimed by Huthis

ROBO SPACE
Microsoft reports strong results driven by cloud and AI; Urges fast 'resolution' of transatlantic trade issues

NASA develops flight-ready aerogel antennas for next-gen airspace communications

British Steel abandons job cut plans after govt rescue

Meta to start using Europeans' data for AI training May 27

ROBO SPACE
EU 'off the pace' in global microchip race: auditors

Intel chief vows to thin ranks at US chip maker

IBM to invest $150 bn in US over five years

MIT engineers advance toward a fault-tolerant quantum computer

ROBO SPACE
Top uranium producer Namibia to open talks on nuclear energy plant

AI driven algorithm streamlines next generation nuclear reactor shielding design

Spain nuclear plants in 'safe' shutdown mode after blackout

Mizzou launches design phase for new nuclear research reactor

ROBO SPACE
US to withdraw some 1,000 troops from Syria

'Bring him home': Philippines migrant workers grapple with Duterte fallout

Iraq says kills senior Islamic State group leader

Iraq repatriates more families from IS-linked Syria camp

ROBO SPACE
How can an electricity network go down in five seconds?

Finland says supports EU goal to cut emissions 90 percent by 2040

UN, Brazil to hold virtual summit Wednesday ahead of COP30

Gunmen attack Chinese-owned power plant site in Chile

ROBO SPACE
UN chief says energy revolution unstoppable despite US pivot

Indonesia says China's Huayou to replace LGES in EV battery project

China's CATL launches new EV sodium battery

Chinese EV battery giant CATL posts 33% surge in Q1 profit

ROBO SPACE
New Shenzhou Crew Begins Handover Operations Aboard Tiangong

Commercial space sector drives China's high-tech ambitions

Veteran Chinese astronaut to lead fresh crew to space station

China to launch new crewed mission into space this week

Subscribe Free To Our Daily Newsletters




The content herein, unless otherwise known to be public domain, are Copyright 1995-2024 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.