California AG sends Musk’s xAI a cease-and-desist order over sexual deepfakes
The flood of AI-generated sexual imagery has spurred concern from state and congressional officials alike.
Earlier this week, the California attorney general’s office announced that it was investigating xAI over reports that the startup’s chatbot, Grok, was being used to create nonconsensual sexual imagery of women and minors. On Friday, the government followed up by sending a cease-and-desist letter to the company, demanding that it take immediate action to stop the production of nonconsensual intimate images and CSAM — child sexual abuse material.
“Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material,” said California AG Rob Bonta in a press release. “The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for [CSAM].”
The AG’s office additionally claimed that xAI appeared to be “facilitating the large-scale production” of nonconsensual nudes, the likes of which are being “used to harass women and girls across the internet.” The agency said it expects xAI to prove that it is taking steps to address these issues within the next five days.
At the heart of the backlash is Grok’s “spicy” mode feature, which xAI created to generate explicit content. The issue has spread beyond California; Japan, Canada, and Britain have opened investigations into Grok, and Malaysia and Indonesia have temporarily blocked the platform altogether. Despite xAI instituting some restrictions on its image-editing features late Wednesday, the California AG’s office moved ahead with its cease-and-desist letter.
X’s safety account has previously denounced this kind of user activity, saying: “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.” TechCrunch reached out to xAI for comment and was met with an automated email that says “Legacy Media Lies.” TC also reached out to the California AG’s office for more context.
The advent of free generative AI tools has led to a disturbing swell of non-consensual sexual material. Many platforms have been grappling with this problem, not just X. The sordid activity has drawn the attention not just of state leaders but also of Congress. Indeed, on Thursday, lawmakers sent a letter to the executives of several companies — including X, Reddit, Snap, TikTok, Alphabet, and Meta — asking how they planned to stem the proliferation of sexualized deepfakes.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
Scale faster. Grow your portfolio. Gain practical expertise. No matter your goal, Disrupt can empower you.Save up to $330 today!
Librarians are hosting viral ‘Avoiding AI’ workshops for people who are fed up with Big Tech
SpaceX launches new V3 Starlink satellites but suffers another booster failure
Prentis, new AI lab co-founded by Reid Hoffman, Mark Pincus in talks to raise $100M
US accuses American of allegedly wiping his phone using a ‘duress’ password during border search
Anduril reportedly in talks to raise funding at $100B valuation, more than 3x last year’s mark
OpenAI makes ChatGPT Health available to all US users
