The online network X has caused major outrage after people used its artificial intelligence assistant Grok to manipulate photos of females by stripping their attire.
Multiple cases have surfaced where the technology has been used to virtually strip women to render them in swimwear non-consensually, as well as to insert them in inappropriate contexts.
XAI, the firm responsible for Grok, remained silent to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to ban image stripping technology. Under a proposed criminal offence, anyone who supplied such technology would "receive a prison sentence and substantial fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with impunity."
The communications watchdog Ofcom said platforms must "assess the potential" of people viewing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' requests when they reference it in a tweet.
It is commonly used to provide commentary or more context to other posters' comments, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification function.
The tool has been faulted for permitting users to produce pictures and footage with explicit content and sexualised content, and it was in the past accused of making graphic material of public figures.
"X has been enabling the creation and distribution of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying images of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or share non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were obligated to take "necessary measures" to "minimize the risk" of users viewing unlawful material on their platforms and to remove it swiftly when they are notified of it.
A wellness coach and writer passionate about holistic health and mindful living, sharing insights from over a decade of experience.