This browser is not actively supported anymore. For the best passle experience, we strongly recommend you upgrade your browser.

The Buzz

Stay informed with the latest research, ideas, news, and more

| 1 minute read

When AI Hallucinates, California Legislates

Law firms, courts and legal regulators everywhere are grappling with the same question: how do we take advantage of generative AI while making sure lawyers remain responsible for the work it helps produce?

In Canada, the response has largely taken the form of professional guidance, court practice directions and law firms developing their own AI policies and guardrails. California has taken a different approach: it has put the guardrails into law.

California moves from guidance to legislation

Senate Bill 574, signed into California law on September 30 and effective January 1, 2027, creates specific obligations for lawyers using generative AI. 

Lawyers remain responsible for AI-generated work

Among other things, lawyers cannot delegate the practice of law to AI and must take reasonable steps to verify AI-generated output and correct erroneous or hallucinated content. 

Confidentiality and verification take centre stage

The legislation also restricts lawyers from entering confidential, identifying and other non-public information into AI systems without appropriate access and confidentiality protections and requires lawyers to verify citations in court filings, including citations generated by AI. Failure to comply can have real consequences, including court-imposed penalties and sanctions.

What Canadian law firms should take from this

For Canadian lawyers, law firms and organizations with in-house legal teams, the significance is less about whether California's particular rules would ever be replicated in Canada and more about the broader trend toward greater accountability for lawyers' use of AI. 
As AI becomes part of everyday legal practice, law firms and in-house legal departments need to think beyond whether lawyers are permitted to use it and focus on the controls around how it is used: protecting confidential information, verifying output, ensuring appropriate lawyer review and maintaining responsibility for the finished product.

AI can assist, but accountability remains human

California may have chosen legislation where Canadian regulators have so far largely taken a guidance-based approach, but the underlying principle is increasingly familiar: AI can assist the lawyer, but it cannot assume the lawyer's professional judgment or responsibility.

Tags

ai, artificial intelligence, legislation, ai-generated work, confidentiality, canada, ai policies