I recently led the reference checking process for a C-level hire for one of our startups. In addition to the multiple reference checks which I conducted, two of my colleagues reached out to their network for additional references.
Any individual reference needs to be taken with a grain of salt. The references that a candidate puts forth are likely to be positively biased because the candidate is unlikely to serve up someone who didn’t appreciate their performance as a reference. Similarly, the references of the people who the candidate worked with extensively which aren’t directly put forth by the candidate are often negatively biased. This is because there’s often a reason why the candidate chose to not serve up someone they worked with extensively as a reference.
As a result of these individual biases, the first few reference checks can leave you confused about what a candidate’s strengths and weaknesses really are. There’s more noise than signal after 2 to 3 reference checks.
This is why you need to enter the reference checking process with the clear goal of getting a holistic and accurate picture of the candidate. Getting 2 or 3 references simply because your company’s HR department has this formal requirement isn’t enough. You need to dig deeper. If you do, you soon start to see the signal separating from the noise. You begin to get a comprehensive picture of the candidate with clear strengths and weaknesses.
I can’t prescribe an exact number of reference checks that are necessary to achieve this. You’ll know when it happens. In this specific case, I checked 4 references myself and we checked over 10 references together with my colleagues. And we only stopped when we felt like we had a clear picture of what the candidate would bring to the table, as well as what they lacked. Each candidate has some of both. If you don’t see both sides that means you need more references.
When you have the candidate’s strengths and weaknesses clearly laid out, it’s a lot easier to see whether their strengths warrant overlooking their weaknesses for the specific role you’re looking to fill.