The online platform X has caused major backlash after individuals used its artificial intelligence assistant Grok to manipulate pictures of women by taking off their garments.
Numerous examples have been seen where the system has been used to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to put them in sexualized contexts.
Company Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to inquiries, except for an auto-reply response stating "legacy media lies".
A government official stated that new laws is being introduced to ban image stripping technology. Under a new law, anyone who distributed such technology would "be subject to a jail term and heavy penalties".
"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people viewing illegal content on their services, but did not specify whether it was presently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which answers X users' queries when they tag it in a message.
It is often used to give reactions or additional context to other users' comments, but people on X are also capable of modify an uploaded picture through its AI image editing tool.
Background of Controversy and Rules
The system has been condemned for permitting users to generate pictures and videos with nudity and sexualised content, and it was earlier alleged to have been involved in making inappropriate content of public figures.
"The platform has been permitting the creation and distribution of these images for months without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying images of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said networks such as X were mandated to take "necessary steps" to "reduce the risk" of users viewing unlawful material on their services and to remove it promptly when they are notified of it.
