The social media network X has provoked widespread backlash after individuals utilized its AI chatbot Grok to manipulate photos of females by stripping their clothing.
Several examples have been seen where the tool has been used to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to insert them in sexualized situations.
Developer Reaction and Legal Position
XAI, the company that created Grok, offered no comment to questions, aside from an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' requests when they tag it in a post.
It is frequently used to give reactions or further background to other users' comments, but people on X are also capable of modify an posted picture through its AI image editing tool.
History of Controversy and Rules
The tool has been faulted for permitting users to create pictures and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in generating graphic material of public figures.
"The platform has been enabling the creation and spread of these pictures for a long time without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own acceptable use policy forbids "depicting images of persons in a pornographic manner".
In a statement, Ofcom said it was illegal to "create or distribute deepfakes or illegal content" and stated this included explicit AI images generated with AI.
It said networks such as X were mandated to take "suitable steps" to "minimize the danger" of users encountering illegal material on their services and to take down it swiftly when they become aware of it.
