A/B Testing in Sourcing to radical change results

Every recruiter/sourcer has it’s own preference when it comes to approaching potential candidates for a career opportunity. Some prefer to call, most choose to send an (in)mail directly to the person, and others prefer to contact someone via a mutual friend or colleague (reverse referral).  But which message gets the best response? A short teaser or a long message with details of the job*? And is the free LinkedIn profile network really a great source or is the paid Job Board CV database more effective?

You probably asked these questions yourself at one point in time.  You can adopt the methods of your mentor or benchmark data as starting point but the best benchmark is your own data! Measuring is knowing.

A/B testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response or conversion rates [source: Wikipedia].

Here’s my advice to get started:

  1. First step is to get a baseline. What has the response been until now? Note: If the response ratio is below 30% you are doing something wrong.
  2. Next, use a tool to track the response. Excel works fine but perhaps your ATS has some functionality build-in to monitor this.**
  3. Set a goal and try to achieve this. For example double the response ratio on Emails by varying the message.***
  4. Start improving the results by making continuous amendments.

Create a talent pipeline report

Example of what kind of information to track when sourcing candidates:

ElementVariables
VacancySpecify Job title
Candidate nameFull name
Source typeSocial network, Job Board, Search company (name generation), Networking, Web search, Employee referral, External referral, Social referral, Reference, Conference, Other
Name of SourceLinkedIn, Monster, SocialCV, Google, Name of referral, etcetera
QualifiedYes/No
ApproachVia Phone, Email, Mutual Connection, InMail, Twitter, FB message, SMS, Reverse referral
Message (A/B test)
  1. Short vs long (100+ words) text,
  2. With/without Job Specification,
  3. Salary/No Salary,
  4. Headline A/B,
  5. Question A/B,
  6. With/without name-dropping, etc
First contactDate
Response dateDate
StatusNot contacted, Contacted, Screened, Applied, Declined, Investigating, Under review, Pending, Interviewing, Assessment, Hired, Do Not Contact
CommentsAny remarks

Do’s and Don’ts

Even though A/B testing is super-simple in concept, keep some practical things in mind.

DON’TS

  • Don’t conclude too fast. Only when you have a lot of data you can judge if source A or message B works better and it takes time to obtain enough statistical information.
  • Don’t compare different messages with different sources as candidate behavior can be very different. A short message (A) may get good results via LinkedIn but may not be the best option to reach out to candidates from Monster. If you want to compare the most effective source than compare the total amount of effort you put in it.
  • Don’t let your gut feeling overrule test results. The winners in A/B tests are often surprising or unintuitive. Your goal with the test is a better conversion rate, so don’t reject the results because of your arbitrary judgment.
  • Don’t forget to include your contact information (phone AND email) and to ask for a response when sending out messages. These are real conversion killers if you. Give it a try in your test…

DO’S

*According to LinkedIn, the likelihood of a response increases by 50% if you mention people you know in common and limit your message to 100 words. Also make it direct and to the point.

** Don’t forget to use your own applicant database for a proactive search.

*** Some sourcers report an 80% response ratio when sending messages to qualified candidates. And when I say qualified I mean qualified i.e. not sending as many standard messages possible to people who mention a keyword on their online profile of CV!

Good luck. Let us know what your approach is and share your best practice!

About Jacco Valkenburg

Jacco Valkenburg is an international recruitment expert, trainer and author (5 books). He has more than 20 years experience in global recruitment strategies and execution spanning numerous countries for leading companies. As founder of Recruiter University and Recruit2 he provides companies with recruitment training and consultancy. His mission is helping companies ‘from good to great staffing’.