The Equal Employment Opportunity Commission (EEOC) and the Department of Justice (DOJ) have issued guidance regarding the dangers of discriminating against individuals with disabilities in the use of artificial intelligence software in hiring and monitoring employees. The EEOC’s guidance is based on its interpretation of the Americans with Disabilities Act (ADA).
The EEOC Guidance
In guidance titled “The Americans with Disabilities Act and the Use of Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees,” the EEOC advises employers to avoid inadvertently discriminating against the disabled by setting software, algorithms, and AI to be used only by those who are not disabled. As one example, the guidance mentions “video interviewing software that evaluates candidates based on their facial expressions and speech patterns.” An applicant with a speech impediment who is fully qualified to perform the job could be discriminated against by such software that automatically disqualifies certain candidates based on their speech patterns.
Software used in the workplace includes “automatic resume-screening software, hiring software, chatbot software for hiring and workflow, video interviewing software, analytics software, employee monitoring software, and worker management software.” All of these are susceptible to built-in systems that make it difficult or impossible for disabled workers to satisfy screening requirements. For example, a blind candidate or employee may be unable to use a given piece of software if the application does not include translation of written words to speech.
As for algorithms, they “process data to evaluate, rate, and make other decisions about job applicants and employees.” They are often part of computer applications and are an unavoidable process incorporated in those applications. As with the software into which algorithms are incorporated, they can pose difficulties to disabled applicants or workers, and thus violate the Americans with Disabilities Act (ADA).
Finally, artificial intelligence or AI (defined as a “machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments”) can violate the ADA if it dismisses applicants or gives employees a lower evaluation based on criteria that discriminate. The EEOC defines disability discrimination thusly: “Disability discrimination occurs when an employer or other entity covered by the Americans with Disabilities Act, as amended, or the Rehabilitation Act, as amended, treats a qualified individual who is an employee or applicant unfavorably because he or she has a disability.”
Screened Out
Employment-related software, algorithms, or AI can “screen out” qualified candidates with disabilities by making it more difficult or impossible for them to complete the screening process. The software can also screen out applicants that can perform the job with reasonable accommodation. “A disability could have this effect by, for example, reducing the accuracy of the assessment, creating special circumstances that have not been taken into account, or preventing the individual from participating in the assessment altogether.” The assessment software should take into account any reasonable accommodation for a disability. Since there are a wide variety of disabilities, the software should be able to take a wide variety of disabilities into account.
Employers should therefore be wary of pre-employment tests. The EEOC writes that “if an employer administers a pre-employment test, it may be responsible for ADA discrimination if the test discriminates against individuals with disabilities, even if the test was developed by an outside vendor.” Furthermore, “employers may be held responsible for the actions of their agents, which may include entities such as software vendors, if the employer has given them authority to act on the employer’s behalf.” As the DOJ’s guidance puts it: “Employers must avoid using hiring technologies in ways that discriminate against people with disabilities. This includes when an employer uses another company’s discriminatory hiring technologies.”
One way an employer can stay in compliance with the ADA is to offer reasonable accommodation during the hiring and work process. “For example, if a hiring process includes a video interview, the employer or software vendor may tell applicants that the job application process will involve a video interview and provide a way to request a reasonable accommodation” if a video interview is somehow impossible for someone with a disability. This is known as a promising practice, and it stays in compliance.
An applicant need not specifically request a reasonable accommodation. Instead,
If an applicant or employee tells the employer that a medical condition may make it difficult to take a test, or that it may cause an assessment result that is less acceptable to the employer, the applicant or employee has requested a reasonable accommodation. To request an accommodation, it is not necessary to mention the ADA or use the phrase “reasonable accommodation.”
Therefore, application and assessment software should allow for users to request a reasonable accommodation, unless to do so would cause the employer undue hardship. If an accommodation goes beyond reasonableness and causes an employer undue hardship, that accommodation may legally be denied.
The DOJ Guidance
The DOJ guidance echoes many of the points made in the EEOC guidance. For example, the DOJ guidance states that:
If a test or technology eliminates someone because of disability when that person can actually do the job, an employer must instead use an accessible test that measures the applicant’s job skills, not their disability, or make other adjustments to the hiring process so that a qualified person is not eliminated because of a disability.
The DOJ guidance uses as an example a visually impaired applicant being given a test that involves seeing. If the applicant can do the job without seeing, the test should also include an alternative version that allows a visually impaired person to perform the test.
What would compliant software do? It will tell applicants about the type of technology being used and how the applicants will be evaluated. It will provide enough information to applicants so that they may decide whether to seek a reasonable accommodation. Finally, it will implement clear procedures for requesting reasonable accommodations and making sure that asking for one does not hurt the applicant’s chance of getting the job.
An employer can run afoul of the law without meaning to. For example, an AI program that compares applicants to successful employees may be discriminatory, because the disabled have historically been discriminated against in hiring and may therefore be underrepresented among “successful employees.” Employers should be cautious when creating or using software in hiring and evaluation.
Conclusion
As the use of software, algorithms, and AI increases among employers, they should be aware of the potential pitfalls of such systems in the inadvertent discrimination against the disabled. Reasonable accommodations can be provided, and applicants and employees can be fully informed regarding how tests and software will be applied.
