Elon Musk's online platform X has sparked widespread outrage after people used its AI chatbot Grok to manipulate photos of females by removing their attire.
Multiple examples have emerged where the technology has been used to virtually strip women to make them appear in bikinis without consent, as well as to place them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the company that created Grok, remained silent to questions, other than an auto-reply message stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to outlaw nudification technology. Under a proposed criminal offence, anyone who supplied such technology would "be subject to a jail term and heavy fines".
"X or Grok could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some additional features—which answers X users' queries when they tag it in a message.
It is frequently used to provide reactions or additional context to other posters' comments, but people on X are also capable of modify an posted image through its AI image editing tool.
History of Controversy and Rules
The chatbot has been faulted for permitting users to create images and clips with nudity and sexualised content, and it was earlier accused of creating inappropriate content of celebrities.
"The platform has been allowing the creation and distribution of these images for a long time without taking any measures and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "create or share non-consensual intimate images or illegal content" and clarified this included sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable steps" to "lessen the risk" of users viewing illegal content on their platforms and to remove it quickly when they become aware of it.
