ARTIFICIAL INTELLIGENCE
🌎 Algorithmic Hiring and Human Judgment: My Firsthand Experience With Hybrid Recruiting in Leeds, UK
Replacing humans with AI in hiring doesn't remove bias. It just changes who gets blamed for it.
For some months I have shifted my focus on fairness and biases in recruitment, reading and reviewing many papers and frameworks. But this time I had my first personal experience with a Hybrid Hiring Model.
I walked into a hotel interview in Leeds, UK and found myself in a room with 15 other candidates. The General Manager walked in and told us something I wasn't expecting, their system had shortlisted everyone in that room algorithmically based on CV parameter matching, but she refused to let it make the final decision. She knew her team. She knew the soft skills, the personality, and the collaborative style that would actually work. So she designed a three-stage process (i) company introduction, (ii) individual assessment, and (iii)group teamwork to evaluate what no algorithm could.
I didn't get the role. But I walked away more convinced than ever that this is exactly what fair hiring looks like.
That experience sent me straight to this research.
I have been reviewing a new March 2026 study by Mesut Kaya from Jobindex A/S and IT University of Copenhagen, and Toine Bogers from IT University of Copenhagen. It is the first study to quantify gender fairness across three real-world recruiting scenarios using 27 months of data from Denmark’s largest job portal.
The research asked a question nobody had answered empirically until now: what actually happens to fairness when you combine human judgment with AI recommendation in hiring?
Here is what they found.
The study analysed 58,765 real recruitment jobs at Jobindex, Denmark's largest job portal and recruitment agency. Recruiters operated in three distinct scenarios (i) searching manually without AI, (ii) relying on AI recommendations alone, or (iii)combining both. Every candidate interaction was tracked across 27 months, measuring gender fairness at each stage using Conditional Demographic Parity scores.
Key Findings
⚖️ AI Alone Is the Least Fair: Pure AI recruiting produced the lowest gender fairness scores, with female candidates consistently underrepresented a CDP ratio of just 0.699 against a perfect score of 1.0.
🧠 Humans Are Fairer Than Algorithms: Human-only recruiting outperformed AI-only with a contacted candidate fairness score of 0.813, suggesting human deliberation naturally corrects some bias that algorithms amplify.
🔗 Human Plus AI Is the Fairest of All: The hybrid model produced the highest fairness score at 0.854 which significantly better than either approach alone, and continuing to improve across the 27-month study period.
🎯 Deliberation Drives Fairness: The more carefully a recruiter evaluated a candidate viewed, clicked, then contacted the fairer the final shortlist became. Scrutiny is the mechanism of correction.
📊 Bias Persists by Job Category: In 85% of job categories, male candidates were overrepresented among contacted candidates including in female-dominated fields like childcare and public administration.
Why It Matters
For HR Technology Developers: Fully automated hiring pipelines are measurably less fair. Human oversight must be built into the workflow structurally, not offered as an optional feature.
For Recruiters and Talent Teams: Engaging with AI recommendations before manual searching actively improves fairness. The Leeds GM I observed understood this instinctively — she used the algorithm to manage volume, then brought her human judgment in to make the call that mattered.
For Policymakers: This research provides empirical grounding for mandating human-in-the-loop requirements in AI hiring tools, giving regulators concrete evidence to justify intervention across the UK, EU and beyond.
For Everyone: AI bias in hiring is not a bug to be patched. It is a structural outcome of training on historical inequality. Human oversight is not the problem — it may be the solution.
"Let's Make Algorithms Work for Everyone. Human-in-the-Loop is a Must."
Paper: Read More
