A 90-person software company has grown quickly and hired badly. Three of the last eight engineering hires left or were let go within a year. The founder, who still interviews every candidate, describes her method as “a proper conversation, I can tell in ten minutes”. The head of people has suggested scorecards twice and been told that the company does not want to turn into a bank.
After the third departure the two of them sit down with the hiring files. They find that the founder’s notes for the people who failed and the people who thrived are almost indistinguishable: “smart”, “good vibe”, “hungry”. There is nothing in the record that would have let anyone tell them apart in advance. The conversation felt rich at the time. On paper it produced no information.
The argument between intuition and structure in hiring is one of the oldest in applied psychology, and it is unusual in that it has a clear answer. When interviews are compared on how well their ratings predict later job performance, the structured kind wins by a wide margin, and has done in study after study since the first reviews in the early twentieth century.
Yet most interviews in most organisations are still unstructured. This guide looks at the evidence, at the psychology of why managers hold on to gut feeling, and at how to build a structured interview that does not feel like an interrogation.
Structure roughly doubles accuracy.
The best current estimates put structured interviews at about .42 validity against .19 for unstructured ones.
The enemy is noise as much as bias.
Different questions, standards and moods make unstructured ratings inconsistent before bias even enters.
Delay judgment, do not ban it.
Score answers one at a time and form an overall view only at the end.
What does “structured” actually mean?
A structured interview has three features. Every candidate is asked the same core questions, in the same order. The questions are derived from the requirements of the job. And answers are scored against a pre-agreed scale, one question at a time.
Structure is a matter of degree. Asking roughly the same questions without scoring them is a small improvement. Scoring answers against written anchors, with interviewers rating independently, is the full version. What structure does not mean is reading from a script in a monotone or forbidding follow-up questions. Probing is allowed and useful, provided it is aimed at understanding the answer, not at rescuing a candidate you like.
How strong is the evidence?
Strong, and consistent across a very long period. Selection researchers measure “validity”: the correlation between a selection score and later job performance, where zero means the method tells you nothing and higher is better.
The classic 1998 summary of 85 years of research put structured interviews at a validity of about .51 and unstructured ones at .38. A 2022 re-analysis that corrected earlier statistical assumptions lowered most estimates but widened the gap: structured interviews came out at the top of all methods at about .42, with unstructured interviews at about .19.
Two points are worth drawing out. First, structured interviews did not merely beat unstructured ones; in the most recent analysis they were the best single predictor available, ahead of ability tests. Second, the number of years a candidate has spent in similar work, which is the thing most CV screens select on, predicts almost nothing. What someone did in those years matters. The count does not.
Why does structure work?
Because it removes noise. An unstructured interview varies in ways that have nothing to do with the candidate: the questions change, the difficulty changes, the interviewer’s mood changes and the standard changes. Some of that variation is the biases described in our guide to interview bias. Much of it is plain inconsistency.
Daniel Kahneman and his co-authors use the word “noise” for this unwanted variability and argue that it is as costly as bias and much less noticed. Two interviewers who meet the same candidate and reach opposite conclusions cannot both be right, but most organisations never find out, because they never compare.
Structure also changes what is being measured. Unstructured interviews mostly measure how likeable and fluent a person is in a first meeting. Structured ones can be aimed at whatever the job needs: judgment in a specific kind of situation, depth of knowledge, how the person has handled a real conflict.
Why do managers resist it?
Not because they have read the research and disagree. The resistance is psychological, and it helps to name the parts.
Confidence in a judgment comes from how coherent the story feels, not from how accurate such judgments have been. A vivid interview produces a coherent story, so it produces confidence. Interviewers rarely receive feedback that would correct it: they never see how the people they rejected would have done.
There is also an identity cost. Judging character is something many senior people regard as a personal strength, and a scorecard implies they are replaceable. And unstructured interviews are simply more pleasant. A good conversation is rewarding. Scoring answers is work.
Researchers who study this resistance, notably Scott Highhouse, have found that people consistently overestimate how much an expert’s intuition adds to a standardised procedure. One striking set of experiments by Jason Dana and colleagues found that people who conducted unstructured interviews were just as confident in their predictions when the interviewee’s answers were secretly random. The interviewers made sense of whatever they heard.
Do not ask a sceptical manager to give up their judgment. Ask them to delay it. The deal is: score each answer as you go, and form your overall view only after the last question. Most managers accept that, and most of the benefit lies there.
How do you build a structured interview?
A serviceable one takes an afternoon. The steps are the same for a warehouse supervisor and a finance director.
Start from the job. Ask the people who do it well and their managers what separates good from adequate. Convert the answers into capabilities such as “plans work under uncertain demand” or “tells customers bad news clearly”. If you have not already done a recruitment needs analysis, this is where it pays off.
Write two kinds of question. Behavioural questions ask about the past: “Tell me about a time a supplier let you down the week before a deadline. What did you do?” Situational questions ask about a hypothetical: “A key supplier tells you on Monday they will miss Friday’s delivery. Walk me through your first day.” Both predict performance. Behavioural questions suit experienced candidates. Situational ones are fairer to people early in their careers who have not yet had the chance.
Anchor the scale. For each question, write a sentence describing a weak answer, a good one and an excellent one. Anchors are what stop a 4 from one interviewer meaning a 2 from another.
| Score | Anchor for “tells customers bad news clearly” |
|---|---|
| 1 Weak | Delays or softens the message; customer learns the real position late or from someone else |
| 3 Good | Tells the customer promptly and plainly; explains cause and next step |
| 5 Excellent | As above, and offers options, agrees a follow-up time and checks understanding |
Limit the panel’s freedom at the right moment. Interviewers score alone, immediately after each answer. Scores are submitted before the debrief. In the debrief, discuss the questions where scores differ by two points or more; that is where someone heard something the others missed.
What does structure cost?
Less than people expect. The set-up is a few hours per role family: agreeing the capabilities, writing questions and anchors, and briefing interviewers. After that, structured interviews are usually shorter to run and much quicker to debrief, because the panel is comparing scores on named criteria instead of trading impressions.
The cost that matters is the one being avoided. Estimates of what a failed hire costs vary widely, but once recruitment fees, salary paid during a poor fit, management time and the delay to the work are added up, it is rarely less than several months of the role’s pay. An afternoon of preparation is cheap insurance against that. The questions are also reusable: the second time you hire for the role, the kit already exists and only needs a review.
Does structure put candidates off?
It can, if it is done coldly. Candidates in some studies rate unstructured interviews as more pleasant, because they feel more like being listened to. But candidates also rate structured interviews as fairer, particularly when the format is explained, and perceived fairness is what predicts whether a rejected candidate still thinks well of the company.
A few habits keep the human side intact. Explain the format at the start: “We ask everyone the same questions so we can compare fairly. I’ll be taking notes.” Leave real time for the candidate’s questions. Keep small talk, but keep it outside the scored part. And tell candidates when they will hear back, then meet that date. More on this in the psychology of rejecting candidates.
Structure is not a guarantee of fairness. A question that depends on experiences only some groups are likely to have had, or anchors written around one person’s style, builds bias into the scorecard. Have someone outside the hiring team read the questions before they are used.
Where does intuition still belong?
At the end, and in a limited role. Kahneman’s advice, drawn from his own early work designing selection interviews for the Israeli army, is to score the separate dimensions first and only then allow a global judgment. Intuition formed after disciplined collection of evidence is better than intuition formed in the first five minutes, because it has more to work with.
There are also things a scorecard cannot settle: how to weigh a brilliant specialist against a steady generalist, whether the team can support someone who needs development. Those are judgment calls. Structure does not remove them. It makes sure they are made on comparable evidence.
The founder in the opening example did not stop interviewing. She agreed to five fixed questions, scored before she compared notes with anyone. Six months later she told the head of people that the scores had twice disagreed with her gut, and that in both cases she was glad she had written the scores down first.
Frequently asked questions
What is the difference between a structured and an unstructured interview?
In a structured interview every candidate gets the same job-related questions and each answer is scored against an agreed scale. In an unstructured interview the conversation goes wherever the interviewer takes it and the result is an overall impression. The first produces comparable evidence. The second mostly produces a feeling.
How many questions should a structured interview have?
Eight to twelve core questions in a 45 to 60 minute interview is typical: about two for each of four to six capabilities, with time for follow-up probes and the candidate’s own questions. Fewer capabilities assessed properly is better than many assessed thinly.
Are behavioural or situational questions better?
Both predict job performance. Behavioural questions about real past events tend to work slightly better for experienced candidates and are harder to bluff. Situational questions are useful for junior roles and career changers. Many scorecards mix the two.
Can small companies use structured interviews?
Yes. The method needs a list of criteria, a set of questions and a scoring scale, all of which fit on one page. Small companies arguably gain most, because a single bad hire is a larger share of the team.
Discover more from Kurums | Business Intelligence
Subscribe to get the latest posts sent to your email.