SACRAMENTO, Calif. — A California state lawmaker is cracking down on AI-generated child pornography, as law enforcement grapples with growing reports of predators using artificial intelligence to exploit minors. The crackdown comes as social media platforms, and the tech industry as a whole, face increasing scrutiny over content moderation and the expanding use of artificial intelligence.
State Assemblymember Marc Berman’s bill prohibits the creation, distribution, and possession of any child sex abuse material created by artificial intelligence. Current state and federal laws already criminalize most explicit content depicting kids - but there’s an unexpected loophole for artificial intelligence.
Law enforcement ran into a wall when they tried to pursue these cases.
“We looked at the statute in existing California law, and recognized there was a significant void,” explained Ventura County District Attorney and bill co-sponsor Erik Nasarenko. “To prosecute an individual for possessing, distributing, or in any procuring child pornography - or child sex abuse material - it had to be a real child. A real image.”
Nasarenko, along with several other district attorneys across the state, brought their concerns to Berman last year. The Silicon Valley lawmaker is responsible for legislation targeting political deep fakes and pushing for increased media literacy across California. State lawmakers are considering several bills intent on regulating the new artificial intelligence industry this year.
“Technology is outpacing society in terms of the laws and regulations that we have in place,” said Berman. “It’s important that we try to learn our lessons from the past and try to get ahead of these problems before they explode out into society.”
This new bill expands the state’s penal code to cover this content, even if it does not depict a real person or act. Supporters, like SAG-AFTRA and Common Sense Media, point to how AI’s machine learning models pull datasets from actual recordings and images of real children.
According to Nasarenko, most of this AI-generated content relies primarily on pornographic material that already exists. It is easier and quicker for AI to generate new content using children who were already abused in real life.
“So the inspiration, the motivation, sadly, are real people. Real kids,” said Nasarenko. “And AI takes those images, those likenesses, those depictions, and goes ahead and victimizes these children in perpetuity.”
According to Nasarenko, most of this AI-generated content relies primarily on pornographic material that already exists. It is easier and quicker for AI to generate new content using children who were already abused in real life.
Meanwhile, children and teenagers who have not been sexually abused are still at risk. This technology can create pornography based on photos posted to a parent’s Instagram page, or videos teenagers upload to TikTok.
Berman believes it is far too easy for any child to become a victim.
“There are companies out there scraping hundreds, thousands of public-facing images on the internet,” said Berman. “From social media and from school websites.”
Countless numbers of photos or videos are used to create just one depiction of this explicit content.
“I would argue that you have thousands, or tens of thousands of victims,” said Berman. “I think every child, every photograph that went into the creation of that sexually explicit image of a child - that all those people are victims.”
AB 1831 eliminates the legal distinction between real and fake child pornography, making it possible to prosecute potential predators.
“It doesn’t matter that it (sexual abuse) didn’t happen,” argued Berman. “It still has a real impact on the people that you’re portraying.”It’s disgusting, it’s obscene, and there’s no place for it.”
However, Berman is expecting some challenges relying on the First Amendment. He has already received emails from people opposed to AB 1831.
“There will be opponents who are ‘free speech absolutists’, let’s call them,” said Berman, drawing on previous experience with social media and AI regulations. “I think any legislation in this space is going to be tricky.”
Additionally, Berman indicated some in the tech industry are not exactly enthusiastic supporters of this type of regulation. Many live in the Silicon Valley areas Berman represents.
“Some tech companies, some platforms have reached out,” confirmed Berman. “Some are in cautious support, and some have reached out to express concerns.”
Online, social media platforms are struggling to moderate child sexual abuse content. Berman believes legislators and platforms need to more seriously discuss how to cease the proliferation of child sexual abuse material online. But Berman says only Congress, and not state lawmakers, are capable of passing broader legislation holding social media more accountable.
Nasarenko says his office still receives reports about AI-generated child porn found online, but none of these potential offenses can be prosecuted - even after staff spend hours investigating.
“One was very graphic and explicit. It depicted a prepubescent minor having intercourse with an adult male,” said Nasarenko. “When you look it, untrained, it looks like child sexual abuse material. But this dedicated child exploitation unit realizes that there are small, very minute differences that reveal it is computer generated.”
Schools are already receiving reports of students using this technology to bully and sexually exploit their classmates. Los Angeles Police are now investigating allegations of Beverly Vista Middle School students abusing AI to create and share nude photos of other students. The school is also investigating, but the first statement to parents and students noted there may be no legal consequences for the suspects.
“While the law is still catching up with the rapid advancement of technology and such acts may not yet be classified as a crime, we are working closely with the Beverly Hills Police Department throughout this investigation,” read the BVMS statement. “We assure you that if any criminal offenses are discovered, they will be addressed to the fullest extent possible.”
Without a bill like AB 1831 becoming law, it is unlikely police will discover any criminal offenses in this incident and others like it. Until then, prosecutors, police, and child safety advocates have their hands tied.
After spending six years prosecuting sex crimes in California, this AI-generated content still manages to disturb Nasarenko.
“Oftentimes, predators will possess child pornography,” said Nasarenko. “That becomes a gateway to more heinous crimes.”
Until AB 1831 becomes law, Berman suggests everyone from kids to parents, and even schools should consider more caution when posting photos of minors online,
“This is a good reminder for everybody. Some bad people are out there, using and abusing the images that we find online,” said Berman. “It’s something to keep in mind as we use social media.”