California is on the brink of passing AB 489 and SB 503 to target AI use (and potential misuse) in healthcare. These measures underscore the state’s ongoing push to safeguard patients and increase accountability in the use of AI.
California lawmakers are moving to regulate how generative AI (GenAI) is used in healthcare, citing risks to patient safety, transparency and fairness. Concerns range from AI chatbots impersonating licensed providers to health insurers relying on algorithms for coverage decisions without proper human oversight.
While federal rules on GenAI are still in development, California moved quickly to pass more AI-related bills in 2024 than any other state — many directly targeting healthcare applications, according to CalMatters. Healthcare providers will face GenAI compliance challenges, so they need to be in the know about existing regulations and what regulatory actions are on the horizon.
California AI Healthcare Laws in Effect Since Jan. 1, 2025
California’s Legislature established itself as a regulatory leader for GenAI in healthcare by passing two foundational laws that took effect on Jan. 1, 2025:
Assembly Bill 3030 – GenAI Disclaimers in Clinical Communications: Requires healthcare providers to tell patients when they are using AI, including chatbots or automated tools, for patient communications involving the health status of a patient. The law also requires providers to include contact information for a real human healthcare provider. The goal is to ensure patients are aware when they’re interacting with AI and to protect them from relying on AI tools without proper human oversight.
Senate Bill 1120 – AI Restrictions in Health Insurance Utilization Review: Known as the Physicians Make Decisions Act, this law prohibits health insurers from relying solely on AI or algorithms to determine whether a request from a provider for healthcare services is medically necessary. Instead, a licensed physician must review and approve those decisions. The goal is to ensure that decisions about the provision of healthcare services are consistent with clinical principles and processes.
Violations of these existing laws expose healthcare providers to criminal penalties under the Business and Professions Code, consumer protection lawsuits under California’s Unfair Competition Law, professional malpractice claims for inadequate AI oversight, and disciplinary actions by professional licensing boards. Monetary damages in class action cases could reach millions of dollars.
California Continues to Lead the Charge on AI Healthcare Regulations
Two bills advancing through the Legislature will further reshape GenAI compliance requirements in healthcare and create new categories of legal liability:
AB 489 – Preventing AI “Impersonation” of Healthcare Professionals. Introduced in February 2025, this proposed legislation would extend existing laws that make it a crime to falsely imply you are a licensed healthcare professional. Under the bill, those rules now apply to companies that develop or use AI systems. This means AI or generative AI tools cannot use titles, letters or phrases, such as “doctor,” “dentist” or “therapist,” that could falsely lead patients to believe that AI-generated medical advice, reports or care are coming from a licensed human provider.
Each violation counts separately, creating significant penalties for noncompliance. Oversight and enforcement fall to the appropriate healthcare licensing boards, which can issue injunctions, restraining orders and other remedies. The goal is to protect patients from being misled or harmed by AI systems that impersonate healthcare professionals.
AB 489 passed the State Senate 39-0 on Sept. 4, 2025, and it is now advancing through the State Assembly.
SB 503 – Comprehensive AI Bias Testing and Reporting. This legislation focuses on preventing bias in healthcare AI systems. If it passes, companies that create or use AI tools for clinical decisions or allocating healthcare resources must report on how they identify and reduce the risk of biased outcomes beginning Jan. 1, 2027.
Developers must also show compliance before releasing their systems, and deployers must submit annual reports to the Department of Public Health. Beginning in 2030, developers will be required to undergo independent third-party audits each year, with audit summaries published on the Department of Health Care Access and Information’s website for public transparency. The law makes both developers and deployers responsible for regularly monitoring their systems and taking steps to correct bias that could affect patient care.
The AI Compliance Challenge: Why Healthcare Providers Need Legal Guidance
California’s ever-evolving AI healthcare regulations are complex and must be navigated carefully. Unlike traditional healthcare regulations, AI compliance spans multiple areas of law, including professional licensing, consumer protection, civil rights and emerging technology regulation.
Healthcare providers should immediately work with experienced healthcare technology attorneys to conduct comprehensive AI compliance audits, identifying every AI tool in their organization and assessing regulatory exposure under current and pending laws. Legal counsel should review and revise vendor contracts to ensure appropriate liability allocation and indemnification for AI-related violations.
AB 489 and SB 503 are advancing quickly through the California Legislature. Healthcare organizations that proactively address these regulatory challenges with experienced healthcare attorneys, like those at Duckor, Metzger & Wynne, APLC, will be better positioned to leverage AI’s benefits while avoiding costly violations and liability exposure. Contact us if you have any questions or would like to talk about your healthcare organization’s AI use.