The online network X has caused widespread outrage after individuals employed its AI chatbot Grok to manipulate images of females by stripping their garments.
Multiple cases have emerged where the tool has been used to virtually strip women to make them appear in bikinis without consent, as well as to place them in inappropriate situations.
Company Response and Legal Stance
XAI, the company responsible for Grok, did not respond to inquiries, aside from an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to outlaw image stripping technology. Under a proposed criminal offence, anyone who supplied such tools would "face a jail term and heavy fines".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing illegal content on their services, but stopped short of confirming whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which responds to X users' prompts when they mention it in a tweet.
It is commonly used to provide commentary or more background to other users' remarks, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been criticised for permitting users to generate images and footage with nudity and sexually suggestive content, and it was in the past accused of making sexually explicit material of celebrities.
"The platform has been enabling the generation and distribution of these images for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of individuals in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said services such as X were mandated to take "suitable actions" to "reduce the danger" of users viewing unlawful material on their services and to delete it promptly when they become aware of it.