Different types of tech test available

What different types of technical assessment can I use?

Choosing the right type of technical assessment is really important. 

It’s important to use the right type of challenge at each stage of your hiring process, and now more than ever it’s important to think about candidate experience.

The days of expecting candidates to spend all weekend working in your technical assessment are long gone.  Good software engineers are in high demand and will make choices about which companies they interview with, based on their process. 

This post covers the different types of technical assessment that are available and provides a balanced view of the pros and cons of each.

Things to think about when considering which type of tech test to use.

  • How relevant and informative the results are
  • Candidate experience
    • Time commitment
    • Feedback 
    • Completed at home or face to face / over zoom
  • Turnaround time (how quickly can you process the assessment and get the results back)
  • Impact on overall time to hire

The different types of technical assessment:

1. Automated tech test

These are generally short, algorithmic based tests sent to candidates early on in the hiring process, candidates complete an online test where they write some code in a browser. Candidates are given a problem statement and asked to write code to solve it. The testing platform then runs unit tests against the code to see whether the tests pass.

Alternatively these automated tech tests can take the form of multiple choice questions where the candidate is given a question with multiple answers and has to select the one(s) they think are correct.

Pros

  • Instant results - because these are ‘scored’ by a machine the results go back to the hiring team immediately.
  • They are relatively cheap to run because they are automated by a machine.
  • Because the coding is done in the browser the screen can be recorded which allows for the hiring team to play back the candidate’s key strokes. This allows the team to see if large chunks of code have been copy / pasted.
  • Saves the hiring team doing any form of assessment as this is all handled by a platform such as HackerRank, Codility or CodeWars.

Cons

  • They produce binary pass / fail results - teams generally set a bar (say 70%) and those that fail to hit the bar are out - there is little scope for borderline cases to be considered.
  • They can fail good software engineers. Simple syntax errors can result in unit tests failing, whereas the rest of the code might have been good.
  • They aren’t designed to test a wide range of skills, they tend to focus on algorithmic knowledge.

###

2. Take-home code challenge

These are much more in-depth challenges. Take-home tests are problem solving code challenges which are designed to be more ‘real world’ than automated tech tests. The idea is to create a problem that might be something you would be asked to do if you were working at the company.

The candidate is given a problem and usually started off with some base code to work from. They work on their own machine, in their own IDE and once they are done they either submit their solution to be reviewed via a platform like Geektastic or they email a link to their GitHub repo or attach a zip file containing their code. 

The duration for the challenge will either be capped (usually somewhere between 1 and 3 hours) or it can be open ended. We talk about the pros and cons of time capped and open ended challenges in another post you can read here.

Other than being completed in the candidate’s own IDE the main difference with take-home challenges is they are reviewed by a human. Usually a senior member of the engineering team will look over the submission, run it themselves and evaluate it across a number of key areas. This means that even if there are silly syntax errors the human can fix them to look more objectively at the solution than a machine can.

Pros

  • If they are designed well the challenges are engaging and aligned to the business where the candidate is applying to join. 
  • They allow the candidate to show a wide range of skills including code quality, solution design, test coverage, problem solving, code maintainability. 
  • Candidates prefer to code in their own IDE than a browser 
  • Humans are able to provide a  much more thorough assessment of the individual’s skills compared to a machine running some tests. 
  • Provided the code submission is anonymised the reviewer should be reviewing the code objectively without any bias. 

Cons

  • They can be too long. Some companies are asking candidates to commit too much of their time to the challenge. 
  • Badly designed challenges can be ambiguous and the instructions too broad to allow the reviewers to compare solutions.
  • Without a well constructed rubric (set of reviewing guidelines) there is a risk that reviewers do not align their scores and you end up with some being more generous than others when grading the submissions.
  • Because each review is carried out by a senior member of the engineering team this can lead to delays. This means candidates can be waiting up to a week for the results. This leads to bad candidate experience and missed opportunities where candidates take roles with teams that move faster (this issue can be mitigated by using Geektastic’s team of expert reviewers who carry out the reviews in less than 24hours).
  • Zip files can get caught up in overzealous email spam filters causing submissions to go missing.

3. Pair programming / Technical interview

These are interactive sessions where the candidate and one or more of the engineering team spend 1-2 hours on a video call or face to face session working through a code challenge or series of technical questions. 

They can either be carried out in a browser based environment or a user’s own IDE.

Pros

  • These sessions allow both parties to interact and have the ability to be fluid. This allows the interviewing to focus in on specific areas as the interview progresses. 
  • It allows for both the candidate and the hiring team to get a feel for each others’ personalities. 

Cons

  • They require extensive training on the part of the interviewer to carry out these types of sessions. 
  • Being face to face can increase anxiety and adversely affect someone’s ability to think straight. 
  • There is a risk of bias influencing the outcome of the session because they are face to face and not anonymous.
  • Arranging a mutually convenient time can lead to delays in scheduling the session.

Of course we are biased, we’ve built our platform around take home code challenges but we do believe there is a place for both automated tech screens and pair programming. We sit alongside both of these tools with our clients. Some will use automated tech screening for high volume applications where they need to quickly filter, they then offer those that pass the initial stage a choice to complete a take-home code challenge at home or arrange a face to face pair programming session.

-—-

Most of Geektastic’s library of take-home technical assessments take between 1 hour to 3 hours to complete. We also migrate our clients’ own challenges to Geektastic, these have a similar range.