Intentionally poisoning someone else is never morally justifiable. However, when faced with a persistent office lunch thief, some may resort to petty acts of revenge.
For artists, safeguarding their work from being utilized to train AI models without consent poses a significant challenge. Requests to opt out and implementation of do-not-scrape codes rely heavily on AI companies’ adherence to ethical standards. However, those driven by profit often disregard such measures. Moreover, for most artists, disconnecting from online platforms isn’t a viable option as they depend on social media exposure for commissions and professional opportunities.
Enter Nightshade, a pioneering initiative from the University of Chicago, designed to provide artists with a means to “poison” image data, rendering it useless or disruptive to AI model training. Ben Zhao, the project’s lead and a computer science professor, likens Nightshade to “putting hot sauce in your lunch to deter theft from the workplace fridge.”
Zhao emphasizes that Nightshade isn’t intended as a powerful weapon against AI companies but rather as a demonstration of the vulnerability of generative models. He clarifies that Nightshade signifies that content owners have means to challenge unauthorized training, offering more substantial repercussions than traditional avenues like writing to Congress or voicing concerns on social media.
Zhao and his team’s objective isn’t to dismantle Big AI but to compel tech giants to compensate content creators for licensed work rather than using scraped images to train AI models.
He concludes by emphasizing the importance of consent and fair compensation, stating, “There’s a correct approach to addressing this issue. The crux of the matter lies in consent and compensation. We’re merely providing content creators with a means to assert their rights against unauthorized training.”
Left: The Mona Lisa, unaltered. Middle: The Mona Lisa, after Nightshade. Right: How AI “sees” the shaded version of the Mona Lisa.
Nightshade manipulates the associations between text prompts by subtly altering the pixels in images, thereby deceiving AI models into perceiving entirely different images than what humans observe. These “shaded” images cause models to inaccurately categorize features, ultimately leading them to generate images unrelated to the original prompts. According to a technical paper currently under peer review, corrupting a Stable Diffusion prompt can require fewer than 100 “poisoned” samples.
For instance, consider a painting depicting a cow relaxing in a meadow. “By distorting this association, you can mislead models into believing that cows possess four round wheels, a bumper, and a trunk,” explained Zhao to TechCrunch. “Thus, when prompted to generate a cow, they might produce a large Ford truck instead.”
The Nightshade team offered additional illustrations. While an unaltered image of the Mona Lisa and its shaded counterpart may appear indistinguishable to humans, AI interprets the “poisoned” sample as a cat adorned in a robe rather than a portrait of a woman.
Moreover, instructing an AI to create an image of a dog after training it using shaded images depicting cats results in eerie hybrids bearing no semblance to either animal.
It takes fewer than 100 poisoned images to start corrupting prompts.
Nightshade, developed by the University of Chicago, disrupts the associations between text prompts by subtly altering image pixels, tricking AI models into perceiving vastly different images from what humans see. For instance, shaded samples that corrupt prompts like “fantasy art” also influence prompts related to “dragon” and “Michael Whelan,” a renowned illustrator specializing in fantasy and sci-fi cover art. According to a technical paper under peer review, even a small number of “poisoned” samples can corrupt a Stable Diffusion prompt.
Consider a painting of a cow in a meadow. By distorting the association, AI models might interpret cows as having four round wheels and a bumper, resulting in the generation of a large Ford truck instead of a cow when prompted. Similarly, an image of the Mona Lisa could be misconstrued as a cat wearing a robe by AI trained on “poisoned” samples.
Glaze, another tool led by Zhao, alters how AI models perceive artistic style, preventing them from replicating artists’ unique work. While a person might view a realistic charcoal portrait, AI sees it as an abstract painting, leading to the generation of messy abstract paintings in response to prompts for fine charcoal portraits.
Zhao describes Glaze as a technical defense against predatory AI companies that ignore opt-out requests. Nightshade, while not an “outright attack,” enables artists to combat unauthorized AI training. Despite OpenAI’s opt-out option, Zhao criticizes its lack of enforcement and suggests that smaller companies may still disregard artist requests.
Artists like Kelly McKernan, part of a class action lawsuit against several companies for copyright violations, see Nightshade as a protective measure until better regulations are in place. Nightshade subtly alters images, with most changes invisible to the human eye. However, some alterations may be noticeable on images with flat colors and smooth backgrounds. The tool, available for free download, offers low-intensity settings to preserve visual quality.
While Glaze and Nightshade aim to impose an “incremental price” on unauthorized data scraping, they ultimately seek fair compensation for artists’ work. Getty Images and Nvidia’s partnership sets a precedent, allowing photographers to receive a portion of subscription revenue for training AI models using their content.
Zhao emphasizes the importance of ethical AI research and its beneficial applications in academia and scientific research. However, he opposes the monetization of Glaze and Nightshade, preferring to keep them freely available to artists. Despite funding challenges, Zhao remains committed to leveling the playing field for vulnerable creators in the face of AI exploitation.