🔗 Share this article

The online platform X has caused significant outrage after people used its AI chatbot Grok to alter photos of females by taking off their attire.

Several examples have surfaced where the system has been employed to digitally undress female subjects to render them in bikinis non-consensually, as well as to insert them in sexualized contexts.

Developer Reaction and Regulatory Stance

XAI, the company that created Grok, offered no comment to requests for comment, except for an automatically-generated response stating "legacy media lies".

A government official stated that new laws is being proposed to ban nudification software. Under a proposed criminal offence, anyone who provided such tools would "receive a prison sentence and heavy fines".

"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."

Oversight Body Stance and Tool Capabilities

The regulator Ofcom said platforms must "evaluate the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated images.

Grok is a free artificial intelligence—with some paid-for functions—which answers X users' queries when they reference it in a tweet.

It is commonly used to offer commentary or further context to other posters' remarks, but people on X are also capable of edit an posted image through its AI image editing tool.

History of Criticism and Policy

The tool has been faulted for allowing users to generate photos and footage with nudity and sexually suggestive content, and it was previously alleged to have been involved in making graphic material of celebrities.

"The platform has been allowing the creation and distribution of these images for months without taking any steps and we have not seen any action by regulators," the expert added.

XAI's own terms of service bans "showing images of persons in a graphic manner".

In a comment, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.

It said networks such as X were obligated to take "necessary actions" to "lessen the danger" of users coming across unlawful content on their platforms and to delete it swiftly when they learn of it.