The Problems with Current Interview Practices; Why You Can’t Find Good Senior Developers

As I mentioned above, a few years back some books were published, which described the interviewing techniques that were being employed at companies like Microsoft and Google. Many of the practices discussed in this post were already being used as those companies. And, according to those books, those companies had had great success using those techniques.

Most people in our world are not very good free thinkers. They just look at what the successful businesses and people have done, and they copy that. It never occurs to them to consider why those practices worked for those companies. Or whether those practices actually had anything at all to do with the success of those companies.

Companies like Microsoft, Facebook, and Google are extremely high profile companies. They literally receive thousands of resumes every day. They have access to a huge pool of developers that want, very much, to work for them. Based solely on the numbers alone, those companies can afford to make up whatever arbitrary requirements and use whatever techniques they can think up to "weed out" candidates, and they'll still have an ample pool to choose from.

Most companies do not have that luxury. Most companies post an ad for a Senior C++ or Java Developer and receive, maybe, a handful of resumes. Those companies, assuming they actually want to get their products completed, do not have the luxury of alienating Senior Software Developers.

A company like Microsoft can put candidates through a 3 day, 6 hour a day, interview process, consisting of endless puzzle and coding questions because:

1. If the candidate gets fed up and walks out, there are 100 more candidates waiting behind him.
2. When big companies like Microsoft and Google are looking to hire Software Developers they aren't looking for people who will think independently, people who will question something that doesn't seem right, people who can make decisions on their own. They already have people for that. What they're looking for are people who will do as they're told and not question things. A person with the dignity and self-respect to not spend three days jumping through hoops is not the kind of person they're looking for.

The point of all this is just because certain interviewing tactics work for certain companies absolutely does not mean they are going to work for yours. In fact, in some respects, adopting certain practices that big, high profile companies use is actually going to harm your company and help those companies you're trying to mimic. How so? By discouraging an otherwise ideal candidate from pursuing a position with your company.

Each of the interviewing practices discussed in this post is, in one way or another, degrading, demeaning, humiliating, or otherwise just distasteful to most software developers who have some degree of experience, and who are very good at what they do.

Buy making the interview process as unpleasant as it is these days, we are discouraging good, skilled, senior developers from pursuing new positions. He will restrict himself to only applying for positions which appear to be of particular interest to him. No competent software developer is going to put himself through hours of technical questions and writing source code to solve problems that will never come up in real world development, for a position he is only moderately interested in. And, the unfortunate reality is that the vast majority of all software development projects are, to most developers, uninteresting. That sounds harsh, but consider: even those developers, like myself, who go home at night or on the weekends and write code and design systems purely out of personal enjoyment - their personal projects are rarely similar to what they're working on at their jobs.

So, if a candidate has to choose between being subjected to a humiliating interview process with a high profile company that is able to offer him substantially more in terms of benefits, incentives, and career advancement (by having a high profile name on his resume); or an equally humiliating interview process with a much smaller company that most people have probably not heard of and, probably are not overly impressed with; which humiliating interview process do you think he's going to subject himself to?

Many very good senior developers stay with their current employers simply because they dread having to go through the interview process. I, personally, despise the current interviewing practices that so many companies are using. The last two positions/projects that I took, were, in large part, because the interviews did not include writing on the whiteboard or excessive technical questions purportedly intended to determine my skill level.

Intelligent companies that want to attract good senior developers will try to make the interview process as easy and comfortable for the candidate as possible. They will seek out other ways to determine the candidate's competence. Those that are truly innovative may even go as far as to try something as radical as, oh, I don't know, perhaps calling the candidate's references. The fact that a developer is "senior" means he's been in the industry for some time. If he really is good at what he does then he should have a trail of references. References can also tell you not only if the person has the required skills but whether they actually apply those skills. It's one thing to have knowledge, it's another thing entirely to actually apply that knowledge to your day to day tasks, and to actually do quality work.

I've noticed, lately, that some companies seem to have accepted the online competence test as a standard part of the early stages of their interview process. This is, without question, the absolute worst practice that a company can employ. It guarantees, more than any other, that good senior developers, those that know they are good, and have any level of self-respect will not bother moving forward with the interview process.

Most good developers know that if they did do the test they'd probably do pretty well. It's not concern about looking incompetent that turns them away. If I may speak completely frankly here: being told to complete an online competence test in order to proceed with the interview process is degrading - pure and simple. It's like the interviewer is saying: "We see your experience; we know you have references; we know you have samples of code and designs you've worked on; but we just don't believe you. And, we want you to spend your time proving to us that you're competent, while we go about our business."

Another major problem with online competence tests is that software development is, in large part, an intellectual and creative pursuit. Sure, there are aspects to software development which are absolute, for example language syntax, and object oriented concepts. Online tests that focus on those absolutes are the lesser of the evils. The tests that seem to be the rage right now, those that present the candidate with a problem and require him to design and implement a solution within a limited time, those are the absolute worst tests. They provide no indication of the candidate's skill level or how he solves problems. Many different people have many different approaches to problem solving. For many senior level developers, one of the first steps in their problem solving process is research. Find out if others have already solved the problem and, if so evaluate and determine whether the existing solutions will meet your requirements. That is the responsible approach. Online competence tests do not facilitate such an approach. They are geared toward testing junior level developers - to determine if they have the ability to decompose the problem into smaller, manageable parts then to translate that into source code.

The only times a good, experienced software developer will submit himself to completing an online competence test is a) if he desperately needs a job; or b) if he is extremely interested in working with that company. Most of the time it's going to be (a).

In short, the easiest way to ensure that good, confident, senior developers will not pursue your jobs is to require them to complete an online competence test.

This is, by far, the most common terrible practice currently being used in interviewing software developers. I see no evidence that it provides any indication of the candidate's skill level, and much like the online competence test, it is, to most intermediate and senior developers, an incredibly demeaning experience.

Unlike the online competence test, which the candidate is typically informed prior to it actually being thrust on him, having to write code on the whiteboard is not dropped in the candidate's lap until he is actually present, on-site for the in-person interview. At that point, most people would feel very uncomfortable refusing to participate (most people avoid uncomfortable situations), so he will usually proceed, though begrudgingly and with significant angst. Unfortunately, that angst and the self-consciousness that goes with being asked to solve a problem off the top of your head - a problem which the interviewer already knows the solutions to because he was the one that provided it - is very distracting. It makes it very difficult to focus on dissecting the problem and coming up with a rational solution.

For those reasons alone, the exercise of writing code on the whiteboard fails to provide reliable insight into the candidate's capabilities.

Some interviewer's have said it is useful because it provides insight into how the candidate functions under pressure. I disagree with that reasoning, though, because what the interviewer should be looking for is how the candidate will function in the day to day environment of the team. If the day to day environment is such that developers are required to write code with pen and paper, without the use of any software development tools (e.g. text editor, compiler, debugger) then perhaps. But if that is the day to day environment of the team then there are much bigger issues for the project manager to address than trying to hire a senior developer.

Many interviewers offer the rationalization that they need to be able to determine the developer's competence before they can decide to make an offer. But if the techniques they're using, e.g. writing code on the whiteboard, fail to aid in accurately or reliably determining the candidate's skill level then why keep using them? Since most developers, even junior developers, hate the idea of writing code on a whiteboard, to solve an unfamiliar problem, while the interviewers watch from across the table, then companies looking to hire the best people should do away with this practice.

If an interviewer really want's to get a representation of a developer's skill level in an interview, and really believes that having them write code on the spot is an effective way of doing that, then they should bring a notebook (laptop) or similar machine into the interview and let the developer write the code using the tools that are actually used by the team, in their day to day work. That would provide a much more realistic representation of the developer's skill level and also prove that they are familiar with using the specific tools. If possible, the notebook can be connected to a projector or large screen so the interviewers can observe - if that makes them more comfortable.

This has become a very, very common practice. The interviewer presents, to the candidate, some hypothetical scenario and asks the candidate to work through the problem on the whiteboard. Ideally, the problem, as presented, should be somewhat vague so that the candidate has to probe the interviewer for additional information. The interviewer wants the candidate to think out loud as he works through the problem. Apparently, this practice of thinking out loud is supposed to provide the interviewer with some kind of insight into how the candidate's mind functions. At least, that's what those books say.

In reality, with the exception of possibly helping to identify people who are completely incapable of abstract thought, this exercise usually proves very little. Amongst intelligent, free thinking people (i.e. those that generally make the best engineers/developers), the human mind does not function based on habit and conditioning as much as it does less intelligent, free thinking people. The people that do best with this exercise are those that have a conscious, deliberate process for attempting to solve problems. Unfortunately, those people are not good at solving abstract problems - they're good at forcing problems to be solved using the procedures they're already familiar with.

The best engineers and developers often don't deliberately dissect a problem. The solutions just seem to come to them. In reality, however, their mind is subconsciously breaking the problem into smaller components, while simultaneously considering questions that need to be addressed. They don't explicitly phrase the questions in their mind - often the answers to the questions come to them before they even consciously consider the question.

For that reason, a good developer is incredibly unaccustomed to thinking out loud while his mind is breaking down a problem and evaluating solutions. Therefore, you are asking an otherwise exceptional developer to think in a way which is completely contrary to what seems natural to him.

In school, we're taught to address problem solving in certain, repeatable, consistent ways that have been known to work best for the masses. And when we're first starting out as engineers we often have to consciously think about those steps as we work through problems. But over time those steps become second nature, habitual. For those reasons, this type of exercise may be well suited to young, inexperienced developers, but to an intermediate or senior developer the practice of thinking out loud as we attempt to resolve an issue is nothing more than a distraction that prevents us from actually solving the problem.

The other major flaw with this practice is that for many experienced developers (or for any line of work, really) one of the first steps in solving any problem for which the developer does not already have a known solution, is researching the problem; evaluating the solutions others have come up with to determine whether their solutions will meet your requirements; and deciding whether to implement an existing solution, possibly modifying it to meet your requirements, or to create a new solution from scratch. In the real world, this is how people actually solve problems. So, if the goal of the exercise is to determine the candidate's approach to solving problems, a more reasonable exercise would be to bring a notebook (laptop) to the interview and allow him to also perform such research as part of the exercise.

Some technical interviewers seem to like asking the most obscure, ostentatious programming questions they can think of. A good, senior developer knows what kinds of issues come up regularly and can usually spot these questions immediately. Unless the question is prefixed with an explanation that this is not something that comes up very often (if ever) then such questions just give the impression the interviewer is being pretentious. Chances are the candidate is going to be turned off by such a demonstration. Regardless of how good of a fit he may be you have probably killed his interest in the position.

Sometimes an interviewer may want to ask such questions to determine the candidate's level of personal interest in a given topic. The rationale is that if the candidate is sincerely interested in something, they will seek to learn as much about it as possible. That seems like a reasonable argument. And as long as the interviewer explains that that is why he's asking, then I believe everything would be fine. Unfortunately, the interviewer rarely provides such an explanation and the candidate is left wondering why he is asking about things which the candidate is very confident the team is not using.

In the best case, the candidate may assume that such issues do come up from time to time and find the challenge of dealing with them interesting. But more likely, he's going to assume the interviewer is showing off or, worse, that the interviewer is trying to make the candidate appear less qualified than he is.

An example of such a question would be anything to do with lambdas in C++. I know what a lambda is and I've implemented them in C++ - not in real world code, but just for my own personal knowledge and experience. If an interviewer were to quiz me on lambdas I would be pretty confident his inquiries had nothing to do with the requirements of the position.

Each technical question a developer is asked which he is not able to answer to his own satisfaction increases his level of anxiety, and as he becomes more anxious he becomes less able to provide clear, concise answers - even to questions he would normally be able to answer easily. Anxiety is very distracting.

If you want to find good, skilled, senior developers then the technical questions you ask in an interview should be related to the requirements of the position. If you really want to pick the candidate's brain about technical issues which are not directly relevant to the position you should save them for the end of the interview and explain to the candidate that what follows is just an informal discussion and will not affect the hiring decision.

Senior Software Developers, no matter how good they are at developing software are, after all, professional software developers - NOT professional interviewers.

I recently interviewed with a large, well known company, with whom I had already, previously spent a year working. I had multiple very positive references from supervisors and peers within the company. I'm sure there was also a file in HR which contained performance reviews. The position I was interviewing for was on a different team, but nevertheless, they had access to the actual source code I had worked on while I was with the company previously.

Despite all of that I was still required to go through numerous telephone "screenings" and finally a 6 hour, in-person, on-site interview. At the on-site interview, I was interviewed by 10 different people, in 6 separate rounds. The interview started at 9:30am and lasted until around 4pm.

After that interview, I went into the office of the job I actually had at that time to try to finish some work. Somebody asked where I was all day. I said I was in an interview for over 6 hours. He replied "Jesus, what were you applying for, the CEO?"

A Software Engineer is JUST a software engineer! He is not responsible for setting company policy; he cannot, single handedly determine the success or failure of the business; if he doesn't work out he can be replaced without significant hardship to the company. After two technical screenings on the telephone, what more does a company hope to learn about someone by subjecting them to another 6 hours of technical interviews over the course of a day? This is one of those practices that high profile companies can get away with only because they have so many applicants.

A candidate is at his peak performance shortly after the interview begins. When the interview first commences, the candidate is probably a little nervous, not sure what to expect. You probably start out informally, help him to relax a bit. That moment is the peak of the candidate's clarity. The longer he is grilled with technical questions, writing code on the whiteboard, asked to solve problems off the top of his head, the less focussed and the more agitated he will be. Yet, at the same time, you keep refreshing the interviewers by swapping them out every hour.

The most likely and common outcome of this type of interview process is that by the end of 6 hours, being questioned by 10 different people, the candidate is sure to have rubbed at least two of them the wrong way.

And this again gets back to the question of "Why, if a candidate has excellent references and proof of their skills and the quality of their work, would you still subject them to a frustrating and humiliating interview process?" If the goal is to find good, capable developers, then why would you create processes that would actively discourage those developers from wanting to apply for the positions you're trying to fill?

By the time I had gotten through 3 hours of the 6 hour interview I had already made up my mind that I wasn't interested in the position. Not for technical reasons, but because of being subjected to a 6 hour interview. But, for the sake of being professional, I continued to the end of the process. Because of that 6 hour interview I would never pursue a position with that company again (unless, of course, they were offering me outrageous money - we all have a price).

Interestingly, the first time I interviewed with that same company, the time I actually took the position with them, the on-site interview was only 2 and a half hours, and the questions were all directly related to the work I would be doing. There was no whiteboarding; no problem solving questions. Just direct "Here's some code, can you tell me what's wrong with it?" If that interview had been like this more recent one I would not have taken that position.

I regularly hear from recruiters, project managers, and human resources personnel about how hard it is to find good, qualified senior developers. Yet, these same companies continue to utilize practices that:

1. Fail to accomplish what they're supposedly intended to - namely, to provide insight into the candidate's ability to break down and solve complex problems, and transform those conceptual solutions into actual source code; and
2. Discourage the good, qualified, and exceptional developers from pursuing those positions by making the process uncomfortable, humiliating, and sometimes even blatantly degrading.

As I've mentioned previously, writing code is only a small part of a senior developer's role. What differentiates a "senior" developer from a "junior" or "intermediate" developer is the wisdom and judgement they've gained through their years and quantity of experience.

But I have yet to participate in an interview for a senior developer position where the line of question actually focuses on the candidate's ability to make good design and implementation decisions. Rather than asking a senior developer to write code on the whiteboard, to prove he knows how to implement recursion, interviewers should be asking the candidate to draw an inheritance diagram, in UML, to show how they would model some set of abstractions.

Rather than focusing on whether the candidate knows how to synchronize access to a resource in a multithreaded system, the focus should be whether the candidate can intelligently determine, in a given situation whether it would be more appropriate to use multiple threads within a single process, or to distribute the load amongst multiple processes, potentially across multiple hosts.

Hopefully, none of the companies I will be interviewing with in the near future reads this post!