There is a new form of discrimination that California employers need to avoid – algorithmic discrimination. In layman’s terms, algorithmic discrimination is a condition in which AI-based automated decision tools contribute to unlawful discrimination, including differential treatment or impacts disfavoring people based on various protected characteristics. Employers that use AI-based automated decision tools must be mindful of Assembly Bill 2930, which is making its way through the California Legislature. The proposed bill aims to prevent algorithmic discrimination by providing a regulatory framework for employers and developers.
Automated decision tools are becoming increasingly available to assist employers in making various employment decisions that affect material terms or conditions of employment (i.e. hiring, termination, pay, promotions, etc.). While these tools can save employers time and money, employers must still comply with California and federal laws. AI system tools use algorithms and inputs created by humans, which may contain baked-in biases. Employers are responsible for decisions made by automated decision tools and other AI-based software, even if the tool is designed or administered by another entity, such as a software vendor. And while AI-based software may not intend to unlawfully discriminate, the duty still falls on the employer to ensure the AI used complies with the law.
To prevent algorithmic discrimination, AB 2930 seeks to regulate “automated decision tools” that make “consequential decisions.” An “automated decision tool” is an AI system or service that makes, or is a substantial factor in making, a consequential decision. “Consequential decisions” are decisions or judgments that have a legal, material, or similarly significant effect on an individual’s life in any one of 13 categories, including employment.
The current version of proposed AB 2930 seeks to regulate AI use as follows:
- Impact Assessment Required: Any employer using an automated decision tool will be required to perform an annual impact assessment, which includes a statement of purpose for the automated decision tool, descriptions of what personal attributes and characteristics the tool is measuring or accessing, the method used to access or measure, how the attributes or characteristics are relevant to consequential decisions, the tool’s outputs and how those outputs are used in making consequential decisions, and an analysis of potential adverse impacts. As currently drafted, the proposed bill provides that employers with fewer than 55 employees will not be required to perform this assessment unless the automated system impacted more than 999 people during the previous calendar year.
- Notification to Employees Required: Employers using automated decisions tools will be required to notify all people subject to a consequential decision that the decision was made by an automated decision tool.
- Automated Decision Tool Policy Required: Employers will be required to establish a clear policy providing a summary of the types of automated decisions tools being used and how the employer manages the reasonably foreseeable risks of algorithmic discrimination that may arise therefrom.
- Governance Program: Employers will be required to establish a governance program to oversee compliance with AB 2930.
- Penalties: In addition to a civil penalty of $25,000 per violation involving algorithmic discrimination, AB 2930 could expose an employer to injunctive relief, declaratory relief and reasonable attorneys’ fees, meaning that a violation of the law could have high-cost consequences.
We will continue to monitor developments affecting the use of AI in the workplace, including tracking AB 2930, and we will provide updates as they become available.
If you are a California employer that uses or intends to use AI to aid in employment decisions, including popular software packages such as Fetcher, Workable, SeekOut, Jobvite or Arya, among others, Please contact Duckor Metzger & Wynne, APLC attorneys Rose Huelskamp Serrano at serrano@dmwplc.com and Katherine C. Fine at fine@dmwplc.com for more information.