The online network X has sparked major anger after individuals utilized its artificial intelligence assistant Grok to change photos of women by stripping their clothing.
Several examples have emerged where the technology has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to place them in inappropriate situations.
Company Response and Legal Position
XAI, the company behind Grok, offered no comment to questions, aside from an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit nudification tools. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and significant fines".
"The platform is capable of halting these forms of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people seeing unlawful material on their services, but did not specify whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' prompts when they reference it in a post.
It is commonly used to offer reactions or further context to other posters' remarks, but people on X are also able to alter an uploaded picture through its AI image editing function.
Background of Controversy and Policy
The tool has been criticised for enabling users to produce photos and videos with explicit content and sexualised content, and it was earlier accused of creating sexually explicit material of public figures.
"The platform has been enabling the creation and spread of these pictures for a long time without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules bans "depicting images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images created with AI.
It said services such as X were obligated to take "suitable actions" to "minimize the danger" of users viewing unlawful material on their services and to remove it quickly when they are notified of it.
