This story was originally published by CalMatters. Sign up for their newsletters.
After Nevada County prosecutors repeatedly used generative AI that added mistakes to criminal court documents last fall, a California appeals court has appointed a judge to consider whether to sanction the district attorney.
The decision comes days after Madison Maxwell, a former Nevada County prosecutor who repeatedly filed court documents with mistakes in them after using generative AI, said in a sworn declaration that her supervisor refused to let her immediately report and apologize for those mistakes to the court, defense lawyers, and suspects.
Maxwell filed a declaration in court earlier this month arguing why she should not face sanctions. An attorney who is sanctioned can be forced to pay monetary fines. The California State Bar can also be alerted about the potential misconduct.
Over a two-month period last fall, Nevada County prosecutors filed court documents with falsehoods in them in four separate cases after using generative AI to write or proofread. Maxwell oversaw three of those cases, including one involving a habeas corpus petition to release a man from jail.
A deeper investigation
The judge’s appointment follows direction from the California Supreme Court in January, when it asked the appeals court to consider a deeper investigation into the extent of alleged misconduct and whether it affected cases. The supreme court also left open the door to sanctions against District Attorney Jesse WIlson by overturning a lower court ruling that would have denied them.
Cases in Nevada County stand out because they involve some of the only known instances in California and nationwide where public prosecutors, who often face a higher standard of conduct, have brought arguments before a court with falsehoods because they used AI.
The Nevada County DA’s Office did not immediately respond to a CalMatters request for comment on accusations made by Maxwell or the ruling by the Courts of Appeal.
In her declaration, Maxwell argues that her supervisor, Assistant District Attorney Lydia Stuart, urged prosecutors to use AI yet had no policy or training in place to guide responsible use of the technology. She added that office leadership fostered an environment where “junior lawyers often operated with minimal-if any-oversight.”
She also claimed that brain fog brought on by high blood sugar and said a restructure of the DA’s office led to departures of senior staff, which increased her workload and decreased her connections with more experienced lawyers.
“Even today, Ms. Maxwell cannot say with certainty which errors were caused by her own drafting errors, as opposed to AI, as opposed to the brain fog and other neurological symptoms brought on by her then-unmanageable diabetes,” her sworn statement reads.
In the declaration, Maxwell said that when she used generative AI to edit documents she submitted to courts she didn’t know large language models can insert citations for cases that don’t exist. Maxwell started work at the DA’s office in July 2024, her first job after law school. She resigned in May and now works in the nearby Sutter County District Attorney’s Office.
Four felony cases affected
Nevada County District Attorney Jesse Wilson and Stuart acknowledged that four felony cases were affected by mistakes that bear the markings of using AI. At that time they said an audit of cases dating back 18 months found no pattern of errors attributable to the use of AI outside of these four cases in February.
It was unclear whether the DA’s office has made the audit public, but a group of more than 20 AI experts urged the California Courts of Appeal to order a deeper investigation into what happened in Nevada, a small county in the Sierra Nevada foothills east of Sacramento .
Since Maxwell’s declaration came to light earlier this month, Civil Rights Corps, a nonprofit that helped the Nevada County Public Defender’s Office file habeas corpus and sanction petitions, requested that the Courts of Appeal take Maxwell’s statement into account and argued that it strengthens the case for the court to order an investigation into alleged misconduct and errors in Nevada County after prosecutors started using generative AI.
“An independent investigation is needed to understand the scope of what really happened here and how many people and cases have been impacted,” Civil Rights Corps attorney Peter Santina told CalMatters shortly before the appeal court decision was handed down.
The California Public Defender’s Association filed a brief in support of sanctions which said there’s a need to deter prosecutors because their mistakes are less likely to comecoming to light. It’s bad when generative AI use leads to mistakes in civil cases, but when a prosecutor can represent to a judge why they should hold someone in custody by citing a case that doesn’t exist, “that has serious implications for that defendant and their family,” executive director Kate Chatfield told CalMatters.
A growing concern over AI in court cases
To date independent researcher Damien Charlotin said there have been more than 130 instances of generative AI use leading to court filings with falsehoods including fake cases in California. Most recently that included a lawsuit over coverage involving State Farm over an insurance claim after a fire. Last year a panel of judges issued a historic $10,000 fine for a Los Angeles-area lawyer who used ChatGPT and filed a court document with more than 20 made-up quotes, a sanction which at the time was issued as a warning to lawyers who use AI statewide. Since then, Charlotin’s database shows that judges have issued sanctions in the form of monetary fines to 19 lawyers and people who represent themselves in California.
As judges increasingly issue fines to lawyers for filing documents with falsehoods after using AI, judges and court staff are also beginning to test use of generative AI. That’s prompted concern about pilot projects underway in Los Angeles and Riverside counties and led to mistakes by judges in federal courts.
UCLA School of Law tech experts previously told CalMatters that they suspect court staff, judges, and lawyers who are overworked are most likely to use generative AI but may lack the time and energy to ensure the technology has not made up facts or inserted falsehoods into court filings.
CalMatters Local News Fellow Cayla Mihalovich contributed to this report.
