{"id":538,"date":"2026-01-14T11:24:00","date_gmt":"2026-01-14T11:24:00","guid":{"rendered":"https:\/\/1newsnetwork.com\/?p=538"},"modified":"2026-02-03T08:44:13","modified_gmt":"2026-02-03T08:44:13","slug":"musks-ai-chatbot-under-fire-countries-step-up-oversight-of-grok-over-non-consensual-deepfake-images","status":"publish","type":"post","link":"https:\/\/1newsnetwork.com\/index.php\/2026\/01\/14\/musks-ai-chatbot-under-fire-countries-step-up-oversight-of-grok-over-non-consensual-deepfake-images\/","title":{"rendered":"Musk\u2019s AI chatbot under fire: Countries step up oversight of Grok over non-consensual deepfake images"},"content":{"rendered":"

Indonesia and Malaysia have blocked access to Grok, while regulators and prosecutors in Europe and Asia have launched investigations or issued formal warnings. Officials and regulators warns of the harms of non-consensual sexual deepfakes associated with the AI chatbot <\/p><\/div>\n\n\n

By Seyit Kurt<\/p>\n\n\n\n

ISTANBUL (AA) \u2013 Governments worldwide are stepping up scrutiny of xAI\u2019s Grok, an artificial intelligence chatbot, amid growing concerns that the tool can be used to generate non-consensual, sexually explicit and manipulated images.<\/p>\n\n\n\n

The controversy centres on Grok\u2019s image-generation feature, introduced last year, which allows users to modify real photographs and create AI-generated images. In recent weeks, the chatbot has been shown to respond to prompts that digitally remove clothing from images of people or otherwise alter their appearance into sexualised content, often without the consent of those depicted.<\/p>\n\n\n\n

Authorities in multiple countries say the misuse highlights shortcomings in existing safeguards and moderation systems, raising concerns over privacy violations, human dignity and the protection of children online. Regulators have warned that such content may cause psychological, social and reputational harm, particularly when real individuals are targeted.<\/p>\n\n\n\n

The issue has also intensified broader debates over the responsibilities of AI developers and online platforms, as governments assess whether current digital and criminal laws are adequate to address risks posed by rapidly evolving generative AI technologies.<\/p>\n\n\n\n

From temporary access blocks to criminal investigations, regulatory probes and formal warnings, authorities across Southeast Asia, Europe and Australia have begun taking action in response to concerns surrounding Grok.<\/p>\n\n\n\n

Indonesia on Saturday became the first country to temporarily block access to Grok, citing its alleged use in generating non-consensual, sexually explicit and manipulated images based on real photographs.<\/p>\n\n\n\n

The Communication and Digital Affairs Ministry said the restriction was aimed at protecting women, children and the broader public from harmful AI-generated content.<\/p>\n\n\n\n

\u201cThe government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,\u201d Communications and Digital Affairs Minister Meutya Hafid said in a statement.<\/p>\n\n\n\n

Indonesia\u2019s National Police described the move as a preventive measure while regulators assess whether the platform violates existing laws.<\/p>\n\n\n\n

Alexander Sabar, director general of Digital Space Supervision, said: “Initial findings indicate that there are no specific regulations within Grok AI to prevent the use of this technology in the creation and distribution of pornographic content based on personal photos.\u201d<\/p>\n\n\n\n

\u201cThis risks serious violations of citizens’ privacy and self-image rights,\u201d he added.<\/p>\n\n\n\n

Malaysia imposed a temporary restriction on Grok following misuse of the chatbot to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors, according to the Malaysian Communications and Multimedia Commission (MCMC).<\/p>\n\n\n\n

In a statement posted on X, the regulator said notices sent this month to X Corp. and xAI requesting stronger safeguards resulted largely in responses that relied on user-reporting mechanisms and failed to address the inherent risks posed by the AI tool.<\/p>\n\n\n\n

The restriction will remain in place until effective protections are implemented, the commission said, describing the measure as preventive and proportionate while legal and regulatory processes continue.<\/p>\n\n\n\n

The European Commission on Tuesday called on X to adopt effective measures to shield EU citizens from sexual deepfakes and other harmful AI-generated content on its platform.<\/p>\n\n\n\n

“We’re still in that window where \u2026 now the ball is in their (X\u2019s) court, and \u2026 we strongly encourage them to \u2026 come up with appropriate measures to settle the case, because we want to settle cases for the benefit of our citizens,” commission spokesperson Thomas Regnier told reporters in Brussels.<\/p>\n\n\n\n

Regnier said X has between 60 and 90 days to address the issue, warning that under the EU\u2019s digital and AI rulebooks, platforms could face significant penalties or enforcement measures if they fail to prevent the spread of illegal content.<\/p>\n\n\n\n

European Commission President Ursula von der Leyen also said Monday in an interview with the Italian newspaper Corriere della Sera that she was \u201coutraged that a technology platform allows users to digitally strip people online,\u201d adding that the bloc would act if companies fail to do so.<\/p>\n\n\n\n

Last week, UK Technology Secretary Liz Kendall urged X to urgently address the issue.<\/p>\n\n\n\n

Britain\u2019s online safety regulator Ofcom later opened a formal investigation into X over concerns that the AI tool was being used to generate sexualised imagery.<\/p>\n\n\n\n

Ofcom said such material could amount to non-consensual intimate image abuse, pornography or child sexual abuse material, all of which are illegal under UK law.<\/p>\n\n\n\n

\u201cThere have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people \u2013 which may amount to intimate image abuse or pornography \u2013 and sexualised images of children that may amount to child sexual abuse material,\u201d the regulator said.<\/p>\n\n\n\n

Ofcom said it contacted X urgently on January 5, setting a January 9 deadline for the company to explain what steps it had taken to protect UK users. After reviewing the response, the regulator said it carried out an \u201cexpedited assessment\u201d and subsequently escalated the case to a formal investigation.<\/p>\n\n\n\n

The Paris Prosecutor\u2019s Office expanded an ongoing investigation into X to include Grok in early January after three government ministers and two members of parliament accused the chatbot of generating fake sexually explicit videos, including content involving minors, French newspaper Le Parisien reported.<\/p>\n\n\n\n

The investigation, open since July, was initially launched following complaints accusing the platform and its executives of manipulating algorithms for foreign interference.<\/p>\n\n\n\n

Sarah El Hairy, France\u2019s high commissioner for children, said earlier this month that she was \u201cscandalised by requests made to Grok AI to digitally undress girls and women,\u201d adding that \u201ceven if the image is artificial, the harm is very real.\u201d<\/p>\n\n\n\n

Italy\u2019s Data Protection Authority (Garante) warned on Jan. 8 that the use or distribution of AI-generated images that digitally remove people\u2019s clothing without consent could result in criminal liability.<\/p>\n\n\n\n

The regulator said such practices may constitute serious violations of fundamental rights and privacy, stressing that AI providers must ensure compliance with data protection laws.<\/p>\n\n\n\n

Italian authorities said they are coordinating with Ireland\u2019s Data Protection Commission, the EU\u2019s lead privacy regulator for X, and noted that national legislation has recently been strengthened to address the spread of AI-generated sexual deepfakes.<\/p>\n\n\n\n

In September, Italy amended its criminal code to introduce penalties for disseminating AI-generated sexual deepfakes as part of broader efforts to combat digital abuse.<\/p>\n\n\n\n

German authorities say they are preparing stronger legal measures to combat \u201cdigital violence,\u201d including the misuse of AI tools to manipulate images into sexualised content without consent.<\/p>\n\n\n\n

Anna-Lena Beckfeld, spokesperson for the Federal Ministry of Justice, said the government is working on legislation aimed at making it easier for victims to pursue legal action.<\/p>\n\n\n\n

“It is unacceptable that manipulation is being used on a large scale for systemic violations of personal rights.” Beckfeld said at a press conference on January 9. “We, therefore, want to see stronger measures taken against this through criminal law.”<\/p>\n\n\n\n

In a post on X on January 6, Wolfram Weimer, Germany\u2019s media minister, urged the European Commission to take legal action to stop what he described as the \u201cindustrialisation of sexual harassment.\u201d<\/p>\n\n\n\n

Australia\u2019s eSafety Commissioner said authorities have requested information from X regarding the Grok\u2019s operation.<\/p>\n\n\n\n

The regulator said it has received a small but increasing number of complaints related to AI-generated sexual imagery and warned it could use enforcement powers, including content removal notices, if material breaches the Online Safety Act.<\/p>\n\n\n\n

It also reminded online platforms and AI providers that under updated rules coming into force on March 9, services must prevent children from accessing sexual, violent or otherwise harmful content.<\/p>\n\n\n\n

Canada\u2019s Minister of Artificial Intelligence Evan Solomon said on Jan. 8 that deepfake sexual abuse constitutes violence and that platforms and AI developers must act to prevent such harm.<\/p>\n\n\n\n

Canada has not imposed a ban or formal restriction on Grok, but officials have raised concerns over the issue.<\/p>\n\n\n\n

\u201cDeepfake sexual abuse is violence. We must protect Canadians, especially women and young people, from exploitation,\u201d Solomon said in a post on X, adding that technology companies have a responsibility to stop the spread of such content.<\/p>\n\n\n\n

Solomon said the government is advancing responsible AI policies, including Bill C-16, the Protecting Victims Act, which would amend the Criminal Code to classify deepfakes as intimate images for the offence of publishing intimate images without consent.<\/p>\n\n\n\n

He added that Canada would \u201ckeep Canadians safe by amending the Criminal Code and holding abusers accountable.\u201d<\/p>\n\n\n\n

South Korea\u2019s media watchdog has asked X to come up with measures aimed at protecting minors from sexually explicit content generated by Grok, according to the Yonhap News Agency.<\/p>\n\n\n\n

\u201cWe have asked the operator of X to prevent potential illegal activities on Grok and submit measures to protect teenagers from harmful content, including limiting or managing their access,\u201d Korea Media and Communications Commission said in a statement.<\/p>\n","protected":false},"excerpt":{"rendered":"

Indonesia and Malaysia have blocked access to Grok, while regulators and prosecutors in Europe and Asia have launched investigations or issued formal warnings. Officials and regulators warns of the harms of non-consensual sexual deepfakes associated with the AI chatbot<\/p>\n","protected":false},"author":2,"featured_media":450,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[43,74,73],"class_list":{"0":"post-538","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-uncategorized","8":"tag-ai","9":"tag-elon-musk","10":"tag-grok"},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/posts\/538","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/comments?post=538"}],"version-history":[{"count":1,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/posts\/538\/revisions"}],"predecessor-version":[{"id":539,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/posts\/538\/revisions\/539"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/media\/450"}],"wp:attachment":[{"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/media?parent=538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/categories?post=538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1newsnetwork.com\/index.php\/wp-json\/wp\/v2\/tags?post=538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}