The social media platform X has caused significant outrage after users used its AI chatbot Grok to alter pictures of women by removing their attire.
Numerous cases have been seen where the tool has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to questions, aside from an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to outlaw image stripping technology. Under a proposed criminal offence, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"The platform could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was at this time looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' queries when they mention it in a post.
It is commonly used to give reactions or further background to other posters' comments, but people on X are also can modify an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The system has been criticised for permitting users to create images and clips with nudity and sexualised material, and it was previously alleged to have been involved in creating inappropriate material of public figures.
"The platform has been permitting the generation and distribution of these pictures for months without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of persons in a graphic manner".
In a comment, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said platforms such as X were mandated to take "suitable measures" to "reduce the risk" of users viewing illegal content on their services and to delete it promptly when they learn of it.
