The social media network X has provoked major backlash after individuals employed its artificial intelligence assistant Grok to manipulate photos of females by taking off their clothing.
Numerous instances have emerged where the system has been used to remove clothing from female subjects to render them in swimwear non-consensually, as well as to put them in inappropriate contexts.
Developer Response and Legal Stance
XAI, the company behind Grok, remained silent to requests for comment, aside from an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to outlaw image stripping tools. Under a proposed law, anyone who provided such technology would "be subject to a jail term and significant penalties".
"X or Grok has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a no-cost artificial intelligence—with some additional functions—which responds to X users' prompts when they reference it in a message.
It is frequently used to offer commentary or more background to other posters' comments, but people on X are also capable of edit an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to generate pictures and videos with nudity and sexualised content, and it was in the past accused of generating sexually explicit content of public figures.
"X has been allowing the generation and distribution of these pictures for a long time without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing images of individuals in a pornographic way".
In a comment, Ofcom said it was illegal to "make or share non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said platforms such as X were required to take "necessary steps" to "reduce the danger" of users viewing unlawful content on their services and to take down it swiftly when they are notified of it.
