|
Getting your Trinity Audio player ready...
|
“People have tried to rewrite history by altering content for hundreds of years. This tendency – to create narratives and push them out and create content that supports those narratives – is something that is inherent to the human experience.
But what is different now is that these tools have come in with the dawn of something called machine learning – which was basically an automation of sorts – and now to AI, which is a more intelligent automation. These tools have just become easier and easier to use.
No longer do you have to be an expert in cutting out a photo – physically altering it – or an expert in Photoshop. You can just be a digital citizen who goes online, inserts a couple of lines of text, and gets out a piece of perceptually realistic content. The problem’s always been there. It’s just the scale and the speed and the ease at which we can sort of create these things has really accelerated in the last three to five years.
There are a few big categories that we really see the most catastrophic or consequential harms coming from AI content.
Obviously, disinformation. Disinformation tends to be on the political side. It’s about sowing uncertainty, muddying the waters. Even if you’re pushing a narrative that’s completely falsified… even if people are absorbing that and saying, ‘Oh, I don’t know if this is fully true. I don’t know if I fully believe it.’ It doesn’t matter. It’s still muddying the waters. It’s still polluting the information ecosystem.
People have tried to rewrite history by altering content for hundreds of years. … AI has enabled us to supercharge an existing harm.”
sarah barrington
There are the secondary effects of disinformation. That brings us into something called the liar’s dividend, which is living in a world where we question everything so much that we actually don’t believe anything is real or anything is fake. What does that mean for legitimate news outlets? What does that mean for legitimate photos and audio?
Then, of course, there’s fraud…. The large scale, really famous (case) being the example in Hong Kong a couple of years ago, where a finance worker transferred out $25 million to a deepfake video conference. And then there’s all these small scale ones as well. Like with Taylor Swift advertising Le Creuset products, for example, or influencers falsely advertising some other brand’s content.
A slightly separate category we’re seeing that is really big at the moment is propaganda or wartime disinformation…. It’s very targeted. It’s the stuff you’re seeing of an attack happen(ing) in Tehran and it goes viral online, and then later it turns out that that video was in fact completely fake.
The final big category is non-consensual internet imagery, which is just a disaster. It was already a disaster before AI, and now we’ve just given all the bad people a suite of tools to do the same harms, but do them much, much worse and at a larger scale. AI has sort of enabled us to supercharge an existing harm.
There are things we can do to protect the public and consumers from this tidal wave of AI-generated or misleading content. But right now there’s hesitation to do that. There are two things going on. Historically, tech companies seem somewhat hesitant to spend time and resources developing these kinds of safety measures. The second is that policymakers are very confused because this technology is very new. It’s very hard to understand, it’s very difficult to regulate.
(Authenticating content has) never been easy, but now it’s getting harder and harder and harder. ….There’s no single artifact we can look for and be like, ‘Ah, gotcha.’ And then the tools that we develop – a lot of our time in the lab is developing these automated or algorithmic detection tools – they update very quickly because the model’s getting better all the time. So we’re always trying to develop methods that will have as much longevity as possible, while also recognizing that it’s inevitable that they’re going to be outdated as the AI models get better and better and better. So it’s always kind of an arms race, but that’s part of the job.
What I advise members of the public to do is just take a step back from the content itself. Just ask who, what, when, where, why? Ask yourself the context. Who is posting this? Why are they posting it? How did they post it? Was it social media versus The Washington Post? It takes five seconds. Just asking yourself some context about the content can protect most cases of disinformation and fraud and anything else that might affect us.”
Sarah Barrington is an engineer, AI researcher, and Ph.D. candidate at the University of California, Berkeley, School of Information. Barrington and her Berkeley adviser, Professor Hany Farid, develop techniques that authenticate real content from AI-generated information.
This profile is one of nineteen featured in the “Humans in an age of AI” portrait series. See more profiles here.
Most Popular
Simiz Kai Lelei brings Tongan and Samoan flavors to East Palo Alto and beyondAugust 26, 2026 11:18 am
