Julien Reszka
The study most people cite when dismissing interviews — Schmidt & Hunter's 1998 meta-analysis — found that unstructured interviews explained about 14% of variance in job performance, while work samples explained about 54%. This became the empirical foundation for 'gut feel interviewing is junk.' The conclusion was reasonable. The numbers were not quite right.
In 2022, Sackett, Zhang, Berry and Lievens reanalysed the same literature in the Journal of Applied Psychology. The problem they identified was a statistical correction error: prior meta-analyses applied a direct range restriction correction when most hiring processes produce indirect range restriction — you select candidates on a bundle of factors, not the single predictor being studied. The correct correction changes the estimates across the board.
What the update does not do is vindicate unstructured interviews. The gap between structured and unstructured formats remains large, real, and the most consequential decision you make when designing a hiring process. Structured interviews — where every candidate is asked the same questions in the same order and answers are scored against a rubric before the hiring decision — predict performance roughly twice as well as unstructured conversations where the interviewer decides in the moment what to ask.
The mechanism is not subtle. An unstructured interview optimises for:
- confidence
- articulateness
- physical presentation
- how much the interviewer enjoys the conversation
None of those are job requirements unless the job is 'be likeable in conversations.' A structured interview forces the interviewer to evaluate the thing actually being hired for, because the questions are written in advance and tied to real requirements, and the scoring happens before anchoring bias sets in.
The practical version is simpler than it sounds: write five questions before the interview, tie each one to something the job actually requires, score each answer on a three-point scale before moving to the next, and ask every candidate the same questions in the same order. That is all. It does not require psychometric training or a formal assessment centre. It requires deciding what you are evaluating before you start.
Discussion
Yes. We've been running 45-minute unstructured chats and calling it 'culture fit'. The people who ace those are not the people who do the work. Writing five structured questions this week.
We added a rubric last quarter and immediately had to confront that our 'A players' were scoring C on actual job requirements. The gut was telling us one thing, the rubric told us another.
This is the friction. The rubric is uncomfortable because it makes the disagreement visible rather than letting everyone nod and move on.
The 2022 correction point is new to me. I've been citing the Schmidt & Hunter numbers in every hiring review without knowing they'd been revised. Reading the Sackett paper today.