New York passes law preventing children from using AI chatbots that 'act as companions'

THE GOVERNOR OF New York has signed a new law preventing minors from using AI chatbots that act as companions, in one of the broadest moves yet for restricting consumer-facing models.

The law requires chatbot operators, such as OpenAI’s ChatGPT, to verify the ages of users and restrict the use of certain traits by the artificial intelligence interface if the user is under 18.

If used by a minor, chatbots would not be able to produce any content that could leave its user to believe it “is human, alive, or experiences human emotions,” the law states.

It also cannot generate outputs that make the user think it “has a personal relationship, professional relationship, or an authority figure role,” nor can it frame the content it produces as being a “personal opinion.”

“No longer will minors have access to chatbots that pretend to be their friends, boyfriends and girlfriends,” New York Governor Kathy Hochul said at a signing ceremony Wednesday.

“No longer will chatbots be able to keep sensitive information about a minor’s emotional and physical health,” Hochul added.

Other US states including California have also passed laws restricting chatbots, but the law signed by Hochul imposes some of the broadest language in regulating the tech, prompting backlash from AI companies.

Tech:NYC, a tech industry advocacy group for the sector in New York, criticised the “vague language” of the law.

“Keeping kids safe online should not mean cutting them off from tools that help them learn and prepare for careers,” the group said in a statement.

AI giants OpenAI and Anthropic did not respond to an AFP request for comment on the law.

Common Sense Media, a nonprofit advocacy group that provides guidelines for parents about media suitability for children, celebrated the bill’s passage, calling it “the strongest kids’ AI safety bill in the country.”

“AI companies offer chatbots with features that are dangerous, and even deadly, for kids and teens…Now, those harmful features will be outlawed in New York,” Common Sense Media CEO Jim Steyer said in a statement.