The online platform X has caused significant backlash after individuals utilized its artificial intelligence assistant Grok to alter photos of females by taking off their garments.
Numerous examples have emerged where the system has been employed to remove clothing from female subjects to make them appear in bikinis without permission, as well as to put them in sexualized situations.
Developer Reaction and Legal Position
XAI, the organization responsible for Grok, remained silent to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification tools. Under a proposed criminal offence, anyone who provided such technology would "be subject to a jail term and substantial fines".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some additional features—which responds to X users' requests when they tag it in a message.
It is often used to offer reactions or additional context to other users' comments, but people on X are also able to modify an uploaded photo through its AI image editing function.
Background of Criticism and Rules
The chatbot has been faulted for allowing users to generate photos and footage with nudity and sexually suggestive material, and it was previously accused of making graphic material of public figures.
"X has been allowing the creation and spread of these images for months without taking any steps and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules prohibits "showing likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary actions" to "reduce the risk" of users viewing unlawful content on their platforms and to delete it quickly when they are notified of it.
