AVOID using pre-application questions/filters for job roles (RAP)

  • by

Anti-Pattern

This is a Recruitment Anti-Pattern (RAP).

IDEA: “With online applications, I can force the candidate to answer questions before they’re allowed to submit their CV/Resume; those answers will improve my hiring!”

REALITY: Forced up-front questions usually reduce both diversity and quality.

Background

Using technology, you can force candidates to answer questions (an HTML form on your website, in the ATS, within the job-application page, etc) and make them upload specific documents. Any candidate who fails to do this is blocked from applying for a job – the webpage won’t allow their submission, and it will delete their application if they don’t add the required fields and continue.

This is commonly used for requiring that every job-application includes a CV/Résumé (PDF or Word doc), and a Cover Letter (Word doc or Text), and that the candidate has confirmed that – yes! – they don’t require a Visa to work in your office location, etc.

Beliefs/Expectations

“Each question moulds the incoming candidate pool, making everyone who gets past that question a better person and better fit for our company. If I define the perfect candidate via these questions, then all the people who apply will be perfect.”

“We will spend considerably less time hiring (because every candidate is so good), and we’ll end up hiring higher-quality people (because only the perfect ones will apply).”

“The more documents and questions that I automatically gather from candidates when they apply, the more time I save on our recruitment”

Reality

Forced questions work by shrinking your candidate pool, not by changing the features of the people within it. If you view the candidate pool as a blob (if you average it) then it appears to be changed, but if you look at the individual candidates moving through it you see that no-one changes, they simply continue or drop-out.

The net effect is that you cut-away any candidates that are interesting, unusual, or possess skills you hadn’t expected. This always reduces diversity, but since it is only subtractive (it cannot add new people to the pool), it continuously reduces the quality of people too. Measuring the ‘average’ of the pool, quality may seem to hold steady or improve, but measuring the ‘peak’ (the actual individuals) you’ll see quality can only trend downwards.

In practice these questions also provide a much lower quality of signal than recruiters (especially HR staff, who won’t be managing the hired person) expect. Non-trivial questions often need to be asked again later (e.g. at interview) in order to explore the missing context, losing most of the hoped-for benefit of “saving time”. Meanwhile the trivial questions often attract false answers, especially from candidates especially eager (positive) – or desperate (less positive) – for the job.

NB: I have often seen HR staff object to hiring-teams asking questions in an interview that “were already answered in the job application” (or which “should have been” included in the application form) … with the engineers immediately objecting that the answer given in person (within a dialogue, with context) was fundamentally different to the one already given on paper (where the candidate didn’t have the context, and the hirer couldn’t obtain any nuance).

Driving forces

Force 1: all gates in your process reduce your pool of candidates

Without your questions: all the same people would have applied, but more and a wider variety would also have applied.

If you know precisely – with no room for adaptation or improvement – everything about the person you wish to hire then … only your diversity will suffer. But if you do not have that perfect foresight then … any gates also reduce the overall quality of the candidate pool. They skew your potential hires towards the narrowest possible interpretation of the job and your hiring need, which will remove some of the best candidates due to over-fitting on your part.

(Corollary: the correct time to use these questions is when your Hiring Managers are overwhelmed with high-quality candidates.)

Force 2: Recruitment is a dialogue, not a monologue

In practice, the perfect candidate – who might have applied for your job – is not sitting pre-packaged in a neat little box waiting for you to post your job. If they were, then you wouldn’t need to hold job interviews! You would do everything via an automated form (note: some companies do this (e.g. An automated hiring process favored me getting a job at Amazon in less than 20 minutes); but notice the context and kinds of jobs they use this for – does Amazon’s tech division use such a process?). Instead that candidate will present a facet of themselves, and it’s up to you and your process to interpret, to understand, to explore how well the person behind that facet will fit with the role you’re trying to fill. This is why we have job interviews.

Force 3: Great candidates need to be pitched to

Recruitment is fundamentally a sales process, and in most tech hiring your biggest problem is getting the ‘best’ candidate to know your job exists, to find it, to be attracted to it, to apply for it, and give you the chance to decide whether or not to employ them. All of that happens before the bulk of your recruitment process – the visible parts – get a chance to run. And this RAP causes you to stop there, failing before you started.

With most job-application systems you won’t even know that the candidate gave up part-way through your application process. Most of the HR professionals I’ve worked with had no idea how many candidates they were burning every day because of this. Do you know the number that bailed on your application process? Do you know who they were? Do you know which questions they DID answer, and how attractive those answers were, before they hit the question that ‘cost you the sale’?

Corrective Action

Basic: Start with no pre-application questions beyond the essentials

  • Name (required for identifying the candidate, and for professional courtesy; you don’t address them with “Hi, CoolDude97@gmail.com, how are you today?”)
  • Email address (required for the process to work at all)
  • CV/Résumé (alternatively: LinkedIn profile) (required to start a dialogue: this is the first step in you knowing something about them)

Basic: If/when the first stage of your recruitment pipeline is overwhelmed by volume of candidates – AND those candidates contain too many top/great candidates – start adding pre-application filters/questions. (remember that pre-application questions can only remove candidates, and so unless you have too-many top candidates, you are likely to remove your best candidates once you start asking them)

Advanced: This can be automated – but first you need to understand that recruiting for high-skilled jobs is all about nuance. A seemingly high-value critical question such as “How many years experience do you have in Javascript?” is meaningless without context – for instance: there is “good” javascript and “bad” javascript, and this question has no way of discerning which the candidate has, or how much of each. The automated questions that would genuinely add value to your hiring process are the ones known as Hinge-Point Questions (HPQs), but these are infamously difficult to create. Very few people know what HPQs are or how to craft them.

More Anti-Patterns & Recruiting Advice

Add your email below to get notified when I publish new Anti-Patterns and related advice.

Or contact me directly via LinkedIn or Email