Programming tests are lame. If you’re still giving out multiple choice “quizzes”, maybe with some rather tricky syntax questions or involving archaic language constructs that nobody actually uses in the real world, you need to stop. Think about what you’re really testing for.
What is it that you want to learn?
Do you really need to know if this person has the parameter order memorized for a common library method? Questions of that ilk remind me of so many tests from school…. really only looking at ones ability to memorize the mundane. You want to see if the candidate can memorize bits of information any semi-decent editor will auto correct for them? Or look up in 10 seconds?
Questions allowing only one correct answer… why? Are you grading this? What score does the person need to get to pass? You have so many candidates that a score comparison on this challenge will have any relevance? (hint: NO)
Oh man we have two great candidates here… one scored 17/20 on the test, the other scored 18/20. I guess it’s obvious which one we’re hiring!
As I’ve mentioned before in previous posts, what I like best about programming is the creative freedom it allows to come up with solutions. Obviously this doesn’t apply to all situations… there are certainly cases where something is either right or wrong… black or white.. but let’s keep our thinking on the macro level here. What you want to see from a potential candidate is a big picture. Let them demonstrate their ability to read and understand a problem and put together a solution delivering the desired results.
Giving the same challenge to candidates of all experiences, let’s you more easily compare apples to apples.
Tailoring a challenge to the level you are hiring for, or think the candidate is at, is pandering. “Here’s a challenge full of questions I know you won’t have a problem with. It’s honestly just a formality the bosses asked me to put together, nobody cares about it. One time we caught a candidate who had absolutely no idea what programming actually was with this, but I don’t know how he made it to this stage.”
My favourite challenges are ones where even the most junior candidate can happily take a stab at, while letting more experienced people really stretch their legs and show off. It’s perfectly fine if a challenge isn’t fully completed, you should explain. If it’s over someones head, have them explain what they don’t understand, or what approach they *might* take on it, given their current knowledge. Pseudo-code is almost better than anything functional.
You want to see how someone thinks.
Challenge during interview, or take home?
Some organizations like to put a candidate on the spot, and give them lengthy challenges to whiteboard or talk out during an interview. Not my favourite thing. My preference has always been to provide a take home challenge. Email the candidate the problem description, have them put something together to bring in to the interview.
Giving someone time to digest a problem is a more representative of a real work environment. You also ensure that nerves don’t factor too heavily into the equation!
Write a simulation for the card game WAR. Deck of fifty-two cards, four of each suit from Ace to Kings. Entire deck dealt out to two players evenly, giving them each twenty six. Each turn, both players flip over the top card from their deck. If one card is greater than the other, the player with the highest card takes the pile and adds it to the bottom of their own. If both cards are equal, a WAR is started. Leaving the two cards already in play, each player draws three more cards, the last drawn being the new scores to calculate on. Game play continues until one player runs out of cards and loses.
Something like that, while on the surface appears to be almost laughable in it’s simplicity, can really get complex. The user probably has to model a deck of cards.. as well as work in the game play conditions. You’ll see some data structures, array manipulation, and a simple event loop. Might even get recursive with the whole ‘WAR’ situation! Not to mention handling cases where the user runs out of cards in the middle of a battle.
Has the user structured it as an Object Oriented thing? Relied on a database somehow? Do they have tests? Was it put into a git project? Is the code cleanly and consistently formatted? So much to see here! So much to evaluate! No two answers will be the same!