ChatGPT Questions Are Getting People Arrested, Authorities Say: Experts Break Down What Not to Type

In recent months, authorities say, multiple Florida teenagers allegedly conducted searches on ChatGPT that led to them being in the crosshairs of law enforcement or facing criminal charges.

The Marion County Sheriff's Office announced the arrest in October of a 17-year-old who is accused of lying about having been abducted by four Hispanic men and even shooting himself as part of a ruse that sparked an Amber Alert. 

The sheriff's office alleges they found ChatGPT searches on the boy's laptop about Mexican cartels and collecting his blood without causing pain.

Separately, the Volusia Sheriff’s Office arrested a 13-year-old in Deland, near Orlando and Daytona Beach, who they claim typed "how to kill my friend in the middle of class" into ChatGPT.

The sheriff's office said officers responded to the school on Sept. 26 after a deputy at Southwestern Middle School was alerted by Gaggle, a school-safety platform that scans school-issued accounts and flags concerning content.

According to the sheriff's office, the student, when questioned, said he "was trolling a friend who was annoying him."

"Parents, please talk to your kids so they don't make the same mistake," the sheriff's office said

Internet activity leading to criminal cases is far from new. But the rapid emergence of ChatGPT — as an artificial intelligence tool for millions of people who rely on it for advice, instant answers and information analysis — is a new example of what experts tell PEOPLE can be a thorny subject.

Tamara Lave, a law professor at the University of Miami, says looking for information itself is not necessarily a crime, with some exceptions such as searching for or distributing child sexual abuse material.

"I think part of what's happening is people are concerned by the ubiquity of ChatGPT and the fact that now, people can access information they could not easily access before," Lave says. "I think we're afraid of what's going to happen with that information."

Catherine Crump, a clinical professor at Berkeley Law School in California who specializes in artificial intelligence and technology law, says that of course it's not smart to Google or use ChatGPT to figure out how to further a crime or how to harm someone.

(ChatGPT has separately faced controversy over its response to people with suicidal behaviors, though the platform maintains it is trained to "recognize and respond to signs of mental or emotional distress.")

Crump says it is important for people, especially children, to recognize that ChatGPT is a product and "not your friend."

ChatGPT can be disarming, Crump says, because it is "designed to be obsequious and supportive" and can make a person feel like they are having a private conversation.

AI companies have created guardrails intended to stop users from gaining information on how to do such things as create a bomb or kill someone, and there are other parameters on the kinds of content users can prompt ChatGPT and other such programs to generate — but they are not foolproof.

OpenAI, the company behind ChatGPT, did not respond to a request for comment for this story.

"I do think there's a measure of individual responsibility people need to take. They need to be mindful of what an AI chat is. And it's a word-association machine," Crump says.

At the same time, Crump contends, "these products have been deliberately engineered to treat users as though they are your friend. And so, I also think there's some measure of corporate responsibility here as well."