San Francisco, the self-proclaimed “home of AI,” is taking a bold stand against the most unethical uses of artificial intelligence. City Attorney David Chiu has launched a groundbreaking lawsuit targeting websites that create and distribute nonconsensual AI-generated pornography, commonly known as deepfakes.

In August 2024, Chiu’s office sued 16 websites responsible for generating “nudified” images of real people without their consent. These sites had been visited over 200 million times in just the first half of the year, revealing a disturbing trend of technological abuse.

The legal action has already yielded significant results. Ten of the targeted websites are now shut down or no longer accessible in California, with one company, Briver, paying $100,000 in civil penalties. Notably, Briver’s sites were capable of generating pornographic images of children, highlighting the urgent need for legal intervention.

This lawsuit comes on the heels of a bipartisan bill signed by President Donald Trump criminalizing the online publication of nonconsensual intimate imagery. The tech landscape is evolving rapidly, with open-source AI models making it increasingly easy for bad actors to create lifelike, unauthorized images.

While many of these sites operate under the guise of “entertainment,” they frequently include disingenuous disclaimers that do nothing to prevent actual harm. One site claims users need “prior written permission” to upload images, but implements no meaningful age or consent verification.

Despite the bleak nature of these websites, Chiu remains optimistic about AI’s potential. “While generative AI holds enormous promise,” he stated, “these website operators are engaged in blatant sexual abuse and must be stopped”.

The investigation revealed that some domain registrars have cooperated in taking down these sites, demonstrating a growing awareness of the ethical challenges posed by emerging technologies.

As San Francisco continues to be a global hub for technological innovation, this legal action sends a powerful message: technological advancement must be accompanied by robust ethical standards and protections for individuals.