Automated Employment Decision Tools in the Crosshairs of New Law

Automation is a top priority among organizations of all sizes, and human resources has been an area of focus. Artificial intelligence, data analytics, and related technologies are being used to automate the hiring process and other employment evaluations. Scanning tools can prioritize certain candidates based on text in their resumes. Virtual assistants and chatbots can evaluate candidates’ mannerisms, while “job fit” algorithms can assess a candidate’s likelihood of job success.

These tools can be valuable, but a New York City law now bans their use unless the employer takes steps to minimize the risk of bias. New York City is not alone: Illinois and Maryland have passed laws regulating AI tools. California, New Jersey, New York, Vermont, and Washington, D.C., have introduced bills similar to New York City’s law, while a Massachusetts bill seeks to prevent “dystopian work environments.” The Equal Employment Opportunity Commission has issued guidance on their use. Other jurisdictions are likely to follow.

Employers should prepare now for the impacts of these resume-screening regulations. The New York City law offers a framework for understanding the implications.

What Is an AEDT?

New York City Local Law 144 defines an automated employment decision tool (AEDT) as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making.” In other words, AEDTs use an algorithm to calculate a score, giving little or no consideration to other factors.

Applications that screen applicants, rank candidates, and assess employees for promotion are among the tools that qualify as AEDTs. However, software that merely scans resumes without screening them would not qualify.

AEDTs can be problematic because of the potential for bias. Even seemingly innocuous questions can cause an AEDT to screen out candidates based on their protected class. For example, software that eliminates applicants with gaps in their work history might discriminate against individuals who took maternity leave.

What is a Bias Audit?

To address these concerns, Local Law 144 requires that AEDTs undergo a bias audit no more than one year before use and at least annually thereafter. The audit must calculate the “selection rate” for race/ethnicity and sex categories included in the EEOC’s EEO-1 Report. The selection rate refers to the number of times an individual in one of those categories is chosen to move to the next stage of the recruitment or promotion process. The audit must also determine the “impact ratio” for each category. That’s defined as the average score calculated for individuals in the category divided by the average score of individuals in the highest-ranking category.

The employer, not the AEDT’s developer, must arrange for the bias audit. The bias audit must be conducted by an independent auditor who has no affiliation with the employer or the AEDT vendor or distributor.

The bias audit must use historical data collected by the AEDT during use. If the auditor determines there is not enough statistically significant data to conduct the bias audit, test data or historical data from other employers may be used.

What Are the Notice Requirements?

Employers must publish the date and results of the most recent bias audit in a conspicuous place in their website’s “careers” or “jobs” section. The data must be published 10 days before the AEDT is used and for at least six months after last use. Employers should also be prepared to disclose their AEDT data retention policies on their websites or in response to written requests.

Employers must give candidates and employees at least 10 business days’ notice that an AEDT will be used, either directly or in a job posting. Employers may also post a notice on their website or in a written employee policy handbook. The notice should include the attributes the AEDT will assess. It should also provide instructions on how to request reasonable accommodation or an alternative selection process.

What Are the Penalties for Noncompliance?

Local Law 144 applies to the use of AEDTs under the following circumstances:

• The job is located in a New York City office, at least part-time, or

• The job is fully remote but associated with a New York City office, or

• The employment agency using the AEDT is located in New York City.

Covered employers will be penalized up to $500 for the first violation and up to $1,500 for each subsequent violation. Each day the AEDT is used counts as a violation, and each failure to provide notice counts as a separate violation. There is no cap on these penalties.

Employers should be aware that the use of AEDTs may violate other laws even if they comply with New York City’s law. For example, Local Law 144 does not force employers to stop using an AEDT if bias is discovered, nor does it require compliance with the EEOC’s 80% rule. However, the EEOC has made it clear that an employer may be in violation of Title VII if the AEDT violates the 80% rule. Furthermore, the employer could be subject to discrimination lawsuits given that the results of the bias audit must be published on the employer's website.

Key Takeaways

Employers that plan to use AEDTs should evaluate them carefully and retain an independent auditor. Because the auditor will need data to conduct the audit, employers should develop policies and procedures for collecting and retaining this data and train staff accordingly. Policies and procedures will also be required for notifying candidates and employees and keeping the website up to date.

Local Law 144 requires employers to keep records of any systems or applications used in hiring and promotion processes. Employers should also keep records of how such tools are used and all notices that were provided. Given the shifting nature of laws and regulations surrounding AEDTs, employers would be well advised to keep such records regardless of location.

More than 80% of employers are using some type of AEDT, according to some estimates. A key benefit of HR automation is the ability to streamline recruitment processes, increasing efficiency and minimizing errors. However, employers must take steps to ensure that AEDTs don’t introduce bias into hiring and promotion processes and comply with Local Law 144, Title VII, and other laws and regulations.

Learn More About New Developments in Law

Stay up to date on the most current legal developments in California and the rest of the nation with Purdue Global Law School.

Purdue Global Law School offers an online Juris Doctor if you wish to become an attorney licensed in California. If you wish to advance your legal education but do not intend to become a practicing attorney, you may consider an online Executive Juris Doctor.

Single law courses are also available to help you explore a particular area of law without committing to a full degree program. Request more information today.