Ofcom Investigating X Over Grok AI Image Tool

The UK regulator is investigating Elon Musk’s X over the Grok AI tool, a probe that could lead to a hefty fine or the social media platform even being banned in Britain.

Ofcom has this morning announced it is testing the weight of the nation’s new Online Safety Act over Grok’s Imagine tool, which has generated headlines and an outcry these past few weeks for producing sexualized pictures of women and children.

Ofcom said its probe “follows widespread reports that a Grok model on X is/was being used to generate and share content that may amount to intimate image abuse, CSAM [child sexual abuse material] and pornography that is accessible to children. 

The new Online Safety Act is aimed at giving teeth to the UK government against the U.S. tech giants. In this case, Ofcom could be fined either £18M ($24.2M) or 10% of qualifying worldwide revenue (whichever is greater). If there is repeated non-compliance, Ofcom could make an application to a court to impose an order that would force internet service providers to ban X. The probe is the highest-profile case for the act, which only came into force two years ago.

X is being probed in respect of its duty to carry out a suitable and sufficient illegal content risk assessment, its safety duties around illegal content, duties to carry out a suitable and sufficient children’s risk assessment and safety duties protecting children.

The backlash against Grok Imagine has come after multiple images were shared of women with their clothing removed. These included a 14-year-old Stranger Things actor, who was manipulated by Grok to put her in a banana print bikini, and a presenter who campaigns for privacy rights after intimate photos were shared around her school without consent when she was 15.

The UK government including Prime Minister Keir Starmer has been one of the tool’s most vociferous critics. After Musk altered X’s rules to restrict the tool to paying subscribers, the government called this move “insulting to victims of misogyny and sexual violence.” A government spokesperson added: “What it does prove is that X can move swiftly when it wants to do so.”

Deadline has reached out to X for comment on the Ofcom investigation. Musk last week said: “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.” X has said: “We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.”

The backlash last week did not stop xAI, the parent company of X that created Grok and its image tool, from announcing a $20B funding raise.