Here’s a question I got recently: “Doesn’t it work to ask a candidate what they would do? couldn’t I present a situation to them and ask what they would do?”
My first impulse was to jump into an answer about behavioral interviewing, and explain how idealized situations invite idealized answers where the candidate talks about how their ideal self would respond to the ideal scenario. I wanted to share how digging into specific behavioral situations lets you see how a candidate has responded in the past, so you can see how they’ll likely behave in the future.
My answer fell flat, and it took some introspecting later to understand why: I’d fallen for the exact problem I was describing. I should have gone into specifics and described specific situations in which the answer might sometimes be affirmative. Part of what makes behavioral interviewing so effective is that it digs into specific situations that really happened, rather than exploring how a candidate might behave in a hypothetical situation.
If you’re trying to learn about a candidate’s past behavior as an indicator for their possible future behavior as your co-worker, ask a behavioral question and dig in for specifics. Use behavioral questions to learn about behavior.
If you’re trying to learn about a candidate’s problem solving skills, it’s ok to ask either a behavioral or a hypothetical question.
Here are some examples to illustrate the difference.
When a hypothetical question might mislead you
“Can you tell me about a time when you had a technical disagreement with someone?”
“I can’t think of any.”
“Oh, that’s ok! What would you do if you were to have a technical disagreement with someone?”
“I would listen to the other person’s point of view and try to work with them to solve the problem.”
This sounds great, right? But what do you learn from this answer?
Does it tell you about the candidate’s ability to identify a technical disagreement? will they steamroll over disagreements because they don’t realize their colleague is disagreeing?
Does it tell you how the candidate might view the other person? will they assume the other person is antagonistic? deliberately obtuse?
Does it tell you how the candidate will work to solve the problem? will they try to do so by repeating their point louder? personal attacks? passive aggressive pseudo-agreement?
When a hypothetical question might help you
“Can you tell me about a time when you had to debug a technical problem and how you approached it?”
“I can’t think of any.”
“Oh, that’s ok! Your resume says you’ve been working mostly on webapps. What would you do if you pulled up one of your webapps and noticed it seemed to be running a little slowly?”
“I’d check if my local network connection seemed ok, check if there were any recent changes to the code or infrastructure, and check all the service providers for any outage or service degradation announcements.”
This also sounds great, right? And this time we learned more about the candidate from the answer.
We learned what they consider when approaching an open-ended question. Since the candidate presented a breadth of ideas covering a range of potential problem sources, we got a sense that their first impulse is not to jump to a conclusion and chase down the answer, but to feel out the landscape.
We don’t know what their behavior will be like during an incident response (will they leak stress all over their colleagues?), but that’s not what we’re evaluating here. We’re evaluating their debugging abilities, not their people abilities.
How to stick with behavioral questions
What happens if we really want to evaluate a candidate’s behavioral abilities, though? how do you rescue when the candidate draws a blank or starts an irrelevant story? Let’s go back to that first situation.
“Can you tell me about a time when you had a technical disagreement with someone?”
“I can’t think of any.”
What were you hoping to learn from this question? Here are a few questions I might be looking for answers on when I ask a question like this:
- How proactively do you communicate with those around you?
- What is your reaction when you see negative feelings directed at your work?
- What communication techniques do you use when navigating a disagreement?
- How much do you value your tech solution, your colleague’s opinion, and the user’s need to get the feature?
- Which of the above is most or least important? does that ever change? what other values might you implicitly tell me about?
- What do you do when confronted with a situation in which everyone can’t be happy with the solution?
Based on this, I’d probably keep the focus on communicating and interacting with teammates, and take the interview in that direction while staying focused on behavioral questions to bring out specific situations.
“Can you tell me about a time when you had a technical disagreement with someone?”
“I can’t think of any.”
“Oh, that’s ok! How about the last major project you collaborated with someone else on?”
“Oh, that would be Project X, which we wrapped up last quarter. This was a team of 3 devs and 1 PM, and no official tech lead. The PM had promised stakeholders it was a 6 week project, and that was last summer.”
“Oh wow, sounds pretty tough. What did you do when you realized it would take more than 6 weeks? How were your teammates doing?”
If you relentlessly focus on the attribute you most would like to understand and ask questions that will bring you closer to an answer, you’ll go in the right direction. Keep digging in deeper (as demonstrated in the above example), and use behavioral questions to draw out specifics that will show you more about who your candidate is.
How do I know when I need a behavioral question?
If you want to know how a candidate will behave – how they’ll react, who or what they’ll prioritize, how they will impact your team’s performance and morale – ask a behavioral question that asks for specific situations.
If you want to see a candidate solve a problem, give them a problem to solve.
Sometimes it can be hard to tell what you’re really looking for. Here are a few questions I might ask myself to figure out if I need a behavioral question or a hypothetical question.
- Will the answer reveal what the candidate’s idealized self does? will that tell you what they do in real life?
- Will the answer show the candidate’s problem solving abilities?
- How would seeing the candidate’s thought process in this answer show you how the candidate does against your evaluation metrics?
I might also try listing out everything I’m hoping to learn (like in the example above).
Finally, I might refer to my assessment metrics for determining if a candidate should be hired or not, and might review those attributes and think about how my question must support that attribute.
Once you know what you’re looking for, use the right tool for the job. If you’re looking for how a candidate behaves, ask a behavioral question that brings out a specific situation that you can dig into.