Jakarta: Grok has faced regulatory action in Indonesia after the government moved to restrict access to the AI chatbot developed by xAI, the startup founded by Elon Musk.
Grok has been blocked on the grounds that its image generation features could be misused to create pornographic and exploitative content, including non-consensual deepfakes. Indonesia has become the first country to deny access to the tool over such concerns.
The Ministry of Communications and Digital Affairs has stated that the production and spread of non-consensual sexual content using artificial intelligence violates human dignity and threatens digital safety. Minister Meutya Hafid said that, the government considers sexual deepfakes a serious breach of human rights and a danger to public security in online spaces.
Grok has been under scrutiny after safeguards were found to be insufficient, allowing users to generate sexualised images, including depictions involving minors. In response, xAI has said it restricted image generation and editing features to paying subscribers while working to improve safety mechanisms. The company has acknowledged weaknesses in moderation systems that allowed inappropriate outputs to appear.
Indonesia, which enforces strict laws against obscene digital material, has acted in line with domestic regulations that ban the sharing and creation of content deemed sexually explicit. The Grok restriction follows a broader tightening of controls on artificial intelligence platforms that are seen as capable of producing harmful material at scale.
Grok has also been criticised internationally. In Australia, Prime Minister Anthony Albanese has expressed concern about the use of generative AI for creating exploitative sexual content. Albanese said that, using artificial intelligence to sexualise people without consent is unacceptable and reflects a lack of social responsibility by technology platforms.
The Australian eSafety Office has said reports connected to Grok remain limited but have shown a recent increase, particularly involving the creation of sexualised or manipulated imagery. The watchdog has confirmed that Grok, X and similar services fall under systemic safety obligations, including requirements to detect and remove child exploitation material and unlawful content under the Online Safety Act.
In the United Kingdom, Elon Musk has also faced threats of fines and regulatory action, with discussions around possible restrictions on X if safeguards are not strengthened. Musk has said on X that, anyone using Grok to create illegal material would face the same consequences as if the content were uploaded directly to the platform.
TOP PICKS | NASA cuts space station mission short over medical issue
