Investigators: Zahira Jaser, Digit Associate Fellow; Dimitra Petrakaki, Co-Lead Research Theme 3, Rachel Starr (University of Birkbeck London), and Ernesto Oyarbide-Magaña (University of Oxford).

This project explores how HR departments and technology companies could improve on the interview experience when using automated interviews with job seekers.

Background

The use of artificial intelligence in HR processes is a new, and likely unstoppable, trend. In recruitment, up to 86% of employers use job interviews mediated by technology, a growing portion of which are automated video interviews (AVIs). AVIs involve job candidates being interviewed by an artificial intelligence, which requires them to record themselves on an interview platform, answering questions under time pressure. The video is then submitted through the AI developer platform, which processes the data of the candidate — this can be visual (e.g. smiles), verbal (e.g. key words used), and/or vocal (e.g. the tone of voice). In some cases, the platform then passes a report with an interpretation of the job candidate’s performance to the employer.

The technologies used for these videos present issues in reliably capturing a candidate’s characteristics. There is also strong evidence that these technologies can contain bias that can exclude some categories of job-seekers. The Berkeley Haas Center for Equity, Gender, and Leadership reports that 44% of AI systems are embedded with gender bias, with about 26% displaying both gender and race bias. For example, facial recognition algorithms have a 35% higher detection error for recognizing the gender of women of color, compared to men with lighter skin.

But as developers work to remove biases and increase reliability, we still know very little on how AVIs (or other types of interviews involving artificial intelligence) are experienced by different categories of job candidates themselves, and how these experiences affect them, this is where our research focused. Without this knowledge, employers and managers can’t fully understand the impact these technologies are having on their talent pool or on different group of workers (e.g., age, ethnicity, and social background). As a result, organizations are ill-equipped to discern whether the platforms they turn to are truly helping them hire candidates that align with their goals. We seek to explore whether employers are alienating promising candidates — and potentially entire categories of job seekers by default — because of varying experiences of the technology.

Our ongoing research starts from a particular demographic: young job seekers from different backgrounds. We chose this demographic because they are likely to experience AVIs as one of the first steps in their initial assessments as they search for their first job. Our initial interviews were conducted with 20 job candidates of different races and ethnicities, largely from Great Britain, and with a mix of first-generation university graduates. We asked them to reflect on their behavior, thoughts, and feelings at the time they were shortlisted for an interview, during the interview, and post-interview. We analyzed their understanding of the hiring technologies, and of the interview process as explained by hiring platforms they were instructed to use. We also studied hundreds of pages of documents, websites, and reports used by platforms to share information about the interview process, the data extracted from candidates, and how it was used.

Preliminary findings

This research, albeit in its initial stage, reveals some interesting findings and already offers a plethora of case studies — some illustrated below — that we have used to inform recommendations to employers and hiring platforms. In particular, our research uncovered four keyways young job candidates experience AVIs:

  1. AVIs Are Hard to Understand
  2. Feelings of Humanity are Diminished
  3. AI Technology Is Glorified
  4. AVIs Are Emotionally and Cognitively Exhausting

To find out more about these findings and how HR managers and platforms can Improve the AVI Experience, please check this publication.

Project website and resources

Media coverage