AI-generated evidence is landing people in jail

FLORIDA — We're all trying to figure out artificial intelligence, and when it comes to the courtroom, what's real is proving to have potentially serious consequences.

In fact, one Florida woman says AI was used to help land her in jail.

Watch full report from Katie LaGrone

"I'm thinking, oh my gosh, I cannot do this. The first time in my life I ever thought about killing myself," the Naples woman told Investigative Reporter Katie LaGrone recently.

This woman says her ex-boyfriend created a fake text that he tried to use against her in an ongoing domestic violence case.

According to the woman, she and her boyfriend got into a fight. He accused her of getting physical. She got arrested, charged and was handed down a no contact order.

But a few months later, he accused her of violating that order by communicating with him and sending him a text, which she claimed was fake.

"No one could show the metadata. No one could show the text in its original form. Everything was a screenshot," she said.

The woman's ex denies he created that text. She was eventually cleared of all charges.

In St. Pete, a woman was recently found guilty of using AI to create a photo of a man she claimed burglarized and raped her. St. Pete police say it was all fake. In her admission to the police, she can be seen on video stating, "I lied about it." She went on to say she was dealing with depression and was seeking attention.

The bottom line: AI-generated evidence is here, and it's becoming a challenge for the courts.
"Do we have a handle on this yet?" Reporter Katie LaGrone asked Tampa criminal defense attorney Paul Figueroa.

"Not really, because it's something that we don't know about fully," Figueroa said.

Part of the problem, experts tell us, is that AI-generated images, video and sound have become so good they're getting harder to detect. Right now, there's a proliferation of AI-powered tools that can make anyone appear to say anything.

Perry Carpenter is Chief Human Risk Management Strategist for KnowBe4, a Tampa Bay-based cybersecurity company.

We asked Carpenter to create an AI-video using a portion of Reporter Katie LaGrone’s broadcast report on this issue. See if you can tell the difference between the non-manipulated video and the AI- video below:

"It's no longer seeing is believing. We have a photo, a recording, a video that by itself is no longer strong evidence. You have to know where that video came from, you have to know who recorded it," said Associate Professor John Licato from the USF Bellini College of AI, Cybersecurity & Computing.

Now imagine you're a judge trying to determine someone's fate based on evidence that may or may not be real.

U.S. Magistrate Judge William Matthewman serves on the bench for the Southern District of Florida. He agreed to speak with us about this issue.

"Is it realistic to say that the ultimate consequence of a nefarious use of AI in the legal system is an innocent person goes to prison?" Reporter LaGrone asked.

"I mean, that could happen," Matthewman said.

While some AI use is considered acknowledged or accepted, it's the unacknowledged AI —or the fake, fabricated and completely made up that Judge Matthewman says is most concerning.

"They look much, much better than some of the fakes that were done years ago," he told us.

When asked what he's most concerned about when it comes to AI in the courts, Matthewman said he worries about "any fake audio or video coming into evidence, but I'm also more concerned about the effect it might have on jurors. There might be perfectly legitimate audio and videos introduced, but there may be people on the jury who say, 'Well, I don't trust any of these. I don't trust any audios or videos,’" he said.

Legal experts are working to establish guidelines for acceptable and unacceptable uses of AI in court. The National Center for State Courts has created a guide for judges.

Florida lawmakers are discussing how to handle AI from a legislative perspective while police and prosecutors are being urged to be extra careful and trace all images, video or sound back to its original source.

It's basically flipping the script on that adage, "seeing is believing."

"Seeing alone is not sufficient for believing," Professor Licato said.

In the spirit of AI, we want you to know this story was reported on-air by a journalist Katie LaGrone, but has been converted to this platform with the assistance of AI. Our editorial team verifies all reporting on all platforms for fairness and accuracy.