Hiring Software Engineers Is About More Than Just Coding Skills

What is an organization if not simply a collection of people all working toward shared goals. People are not resources; people are your company. That is why it is so important to get it right. A mediocre employee can destroy the best ideas, and the best employees can make mediocre ideas excel.

Hiring the best is your most important task. — Steve Jobs

This is why it is so important to get this right, but how should you even begin to approach this problem?

How You Should Hire

You can’t assess company and role fit until you first know who you are. I can’t stress this enough. Take the time to figure out your mission; hash out your values; establish your guiding principles. It is much easier to train someone in a specific work related skill than it is to teach them how to have initiative, work autonomously, and value communication. These things are what makes you who you are. They are what help you decide between equally viable “good things”.

At Codecademy, we value learning and a growth mindset. This means we are more likely to take a risk on someone who is early in their career but shows significant potential than someone who is very skilled, but doesn’t play well with others.

Character and values are not the only indicators of success. As an Engineering Manager, you are playing the role of conductor. You need to deeply understand the work at hand, the work coming down the pike, and the current skills on the team.

Each team is unique. Some teams work very autonomously. Some teams work collaboratively. Some focus on backend work while others work on mobile apps. You need to take into account the skills on the team, the position along their learning journey of each member, and working dynamics to make the optimal choice of who to invite onto the team.

It is better to be understaffed for a short time than to make the wrong hiring decision.

The goal of interviewing is first and foremost identifying the person that will be most successful in the role and who will bring the most to the table. This isn’t about finding the smartest. This isn’t about finding the most educated. This is about finding the people who will contribute the most. We should never try to trick people. This isn’t an endurance test. We should design our hiring process to be open, honest, and effective.

Another principle that I would like to instill is that hiring is a continual process. You will always have some degree of turn over, so you want to always be on the lookout for new talent. You don’t want attrition to sneak up on you, as the process takes time to get momentum. Always keep an eye out, and continually refine your processes.

Crafting The Job Description

The first step in creating your hiring funnel is crafting the job description for your hiring post. Here are some things to think about when writing this.

Be honest. I may have to say this a number of times, but it bares repeating. Be honest. You don’t want to spend the time and money to get someone into a role only to have them quit two weeks later because you misrepresented the work. They should know what they are getting into and should be excited about it.

Tune the message to the role. I find that people early in their career get really excited about working with fun new technology. People who are experienced tend to focus on what types of problems they will be working on. They are already effective with a set of skills, and they are looking to do something interesting with them. People with a lot of experience tend to focus on the people and purpose. They want to know why they are solving these sets of problems vs others. They want a broader impact.

Luckily these three personas roughly line up with the “qualifications”, “responsibilities”, and “about us” sections of a job description.

Sell it. People want to be excited about your opportunity. Give them a reason. Enthusiasm is contagious. If you want others to be excited about working for your company, you should be excited about working for your company.

Finding The Right People

Now that you have your job posting in hand, what do you do with it? Hopefully you have internal recruiters in your People Ops department that can help you with this. Even if you do, I would recommend leveraging your networks. You have had years of experience, and have met many along the way. Now is not the time to be shy. Share the posting with your previous colleagues. Ask them to share it with their colleagues. Are you part of any groups or meetups? Do you attend industry conferences? I would even ask your existing team members to contact their networks for you.

One of the hidden traps to leveraging your existing networks is you are likely to get stuck in a local optimum. You are more likely to know people who act, think, and yes even look just like yourself. Your best best to cast a wider net is to partner with existing organizations who already have connections to the community. When it comes to diversity, equity, and inclusion, my policy is that we never lower our standards. Everyone we hire has earned their place with us, but we do make an effort to extend the invitation to those who may not know about us.

Crafting Your Funnel

I call it a funnel for a reason. In many ways a hiring funnel is no different than a signup or purchase funnel. There are a set of steps specifically designed to solve a problem, and we should be aiming to make the process as smooth and painless as possible.

Every funnel should be as unique as the role you are hiring for. I’m going to focus on software engineers for our purposes.

Top of the Funnel

The top of the funnel is not the resumé submission. A passive submission process is a poor and [no offense] lazy way to advertise your role. We wouldn’t do that if we were selling a product, and we shouldn’t do that in finding the best engineers. Our aim is to cast our net as wide as possible. This not only increases the chances of finding top talent, but as previously mentioned, increases the chances of finding diverse talent. Tap your network. Tap the networks of your existing team members. Leverage user groups and community groups. Use advanced LinkedIn searches, and yes even use external recruiters if you don’t have the in-house support you need. Quality is a numbers game.

Screening

This usually has three steps. Hopefully your PeopleOps team will be able to help you with this, as it is infeasible to have a high touch interview with each of these candidates.

Usually this process begins with a resumé review. You should know the role and level you are hiring for, so you (or your recruiter) should be able to scan through the resumes for the amount and type of experience, and the necessary skill sets.

Next up is the intro call. This is often done by a recruiter, as it doesn’t require any special knowledge. This is to assess basic eligibility, interest, and maybe communication skills. It is an opportunity to sell the candidate on the role and company, and answer any questions they may have about the process.

Finally there needs to be some kind of skills/technical screening. As unfortunate as it is, some people lie on their resume, so you need some way to be able to easily assess a fairly large number of people. That brings us to the elephant in the room.

Algorithms and Data Structures

There is a long standing debate in the industry of what is the best way to assess technical skills. There are a number of large tech companies that use complex data structures and algorithms to be able to assess technical proficiency. They do this for a number of really good reasons.

For starters, this screen needs to be straightforward, not waste a lot of time, and be fully self contained. Rarely is software actually written in a vacuum; however, you can’t expect a brand new candidate to have intimate knowledge of existing systems and practices. So familiarity with a complex algorithm that can roughly be written in 400 lines, and have no major external dependencies is really convenient. In addition, these rarely represent any kind of significant novel invention thus side stepping any concerns about garnering unpaid work from candidates.

Finally, it can work. There is a high correlation between people that can do well on those tests and people write complex and elegant code. This is due to the fact that people with high intelligence (specifically problem solving and large working memory) and a high work ethic (as it often takes a long time diligently studying arcane patterns that are unlikely to show up in your work) will often make good programmers.

So that begs the question: Is this a good strategy?

No … not likely. Why? Close is not good enough. Remember back to the principle that we are not trying to trick people; we are trying to assess how successful they will be at the role? Unless the role is writing a compiler for a new language, these types of challenges are just not relevant to the types of problems an engineer will be facing on a day to day basis.

We are builders. We build applications. We design code. We optimize. This is why I prefer coding challenges that require people to build a small application or to refactor some spaghetti code. This will show you how they write tests, how they organize code, and their familiarity with relevant languages and frameworks.

The On-site

Once the huge pool of candidates is reduced down to a reasonable number, now is the time to get them “into the office”. I encourage you to do this in cohorts, so that you always have a pool of candidates to compare. In general, we have roughly three modules.

Coding — This is a hands-on exercise similar to the tech screening coding challenge. This is an opportunity to see if people really know their craft and not just “talk the talk”. Do they know their tools? How do they problem-solve? How is their communication with the facilitator? Can they understand and incorporate suggestions? Are they effective at finding docs and references?

Design & Architecture — For people earlier in their career, coding is more important. For more senior engineers (and managers) software design and architecture is more important. This is the infamous white-boarding session. The important thing is to not confuse this with the coding module. This is not about how well they can code. Face it; coding on a whiteboard is painful. This is about understanding what levels of abstractions they are comfortable with. You want to understand where they gravitate to. Do they understand class relationships? Can they model the database? Do they know the pros and cons of various API designs? What questions do they ask or clarifications they require? Do they organize the work into multiple apps? Is caching involved? What motivates their decisions (some “ideal” pattern or the specific constraints of the problem)? I even like to know if they know how to break work down in a way that can easily be delegated out to a team.

Behavioral — At Codecademy, we always have the hiring manager facilitate this one. This can be a tricky one, as it requires a lot of intuition and empathy. This isn’t a matter of “do the tests pass”. This is about understanding what the candidate is saying and what they are not saying. For this reason, we prefer to have two managers conduct this interview in order to get multiple perspectives. We assess things like their communication skills, leadership skills, how they work in teams, and how they resolve conflict. This is where those clear values I insisted on come into play. Does this person fit into our company and the team? Whether your team is data driven, radically open to feedback, or is highly collaborative, this is your opportunity to make sure they will be successful in your unique environment. It is also an opportunity to get a little feedback about the process and do a little selling of the team and company.

The Huddle

This practice isn’t an interview, but is just as important to the process. Everyone should have written clear scorecards with concrete pros and cons. The hiring manager should have reviewed all the feedback and recommendations before this meeting. This is the final meeting to fill in any gaps. It should be a collaborative discussion with all the people involved in the on-site process. People should ask questions, dig into areas, and get an overall temperature. The hiring manager is responsible for soliciting everyone’s feedback, synthesizing it, and making the final call.

Another principle we try to hold to is, if you are on the fence, the answer is no-hire. This may seem harsh, but there are two concepts that support this.

First, hiring is one of the most impactful and important decisions you can make. There is often an implicit promise of mutual support, and having to manage someone out can be painful and legally precarious. It is better to miss a qualified candidate, then to hire, onboard, and later have to fire a poor candidate.

Additionally, it is unfair to all parties. Not only is your team negatively impacted and the candidate is negatively impacted (it is not fun to feel like you are set up to fail), but it is also unfair to all the other candidates who would have been a good fit. We humans have a subtle bias towards loss aversion. After spending so much time with a candidate, there is a desire to “make it work” for the person in front of you. However, this is unfair to that person next in line who is equally qualified and an even better fit, but simply hasn’t gotten a chance to be interviewed yet.

Reference Checks

The decision of whether to make an offer happens in the huddle, but usually the offer is contingent on reference checks. That being said, I have yet to rescind an offer based on a reference check. For me, the primary purpose of the reference check is to get a head start on a career development plan and understand what environments are best conducive to the candidate’s success.

Measure and Iterate

As with any funnel, it is your responsibility to assess the health and improve the process. In and after the interview, you can solicit qualitative feedback from the candidate. Was the on-site too long? Should we incorporate breaks? Was the test laptop properly configured? Were the interviewers kind and courteous? However, there are some subtle signs of failure that don’t show up until you look at the numbers.

You should understand the passthrough rate at each stage. This might look like 1000 resumes yields 100 technical screens which yields 10 on-site interviews. You should have an idea of what is healthy and maintainable at each stage. Are you getting too many unqualified candidates to the on-site phase? This might mean you need to increase the difficulty of your tech screen. Are there too few that are passing the resume screen? Perhaps your recruiter is putting too much weight on specific languages or frameworks. Look for bottlenecks. Too many or too few could be signs of failure. Also, don’t be afraid to analyze the data segmented by demographics. Though distinctions don’t dictate discrimination, it could point to it. With a large enough sample size, it should be roughly equal.

Who Makes The Call?

At Codecademy, we distribute the interview load among all engineers above a certain level to ensure no team is disproportionately impacted by the responsibility. However, this comes at a cost. Most of our engineers have studied interview modules and rubrics, shadowed and conducted interviews, and received training to recognize implicit bias.

We foster a collaborative culture and strive to incorporate as many perspectives as possible. Nevertheless, we also believe in the concept of a Directly Responsible Individual (DRI). The hiring manager’s responsibility is to consider all views and perspectives, reconcile them, and make the final decision. This decision should align with everyone’s views, but ultimately, the hiring manager will be held accountable for the results.

Some companies feel this approach gives too much power and opportunity for individual bias, and they set up committees for the final round of approvals.

Candidate Experience

At Codecademy, we have the advantage of serving the same population we hire from. This means that many people we hire have either used our product to get started in the industry, or have the potential to influence peers to use it in the future.

For this reason, it is even more important for us to strive to turn every candidate into a brand advocate. We strive to create an environment in which candidates feel comfortable and confident, and where they feel that their time and effort are valued. By treating candidates well and creating a positive experience, we hope to build a reputation as an employer of choice in the tech industry. We believe that every candidate, regardless of whether or not they are ultimately hired, should leave the interview process feeling respected and valued.

Soliciting feedback is an essential part of any hiring process. It is important to understand the candidate’s experience and to use that feedback to improve the process. Feedback can help identify areas where the process could be streamlined or improved, and it can also help identify potential biases or other issues that may be impacting the candidate experience. By using feedback to continuously improve the hiring process, we can create a more positive and effective experience for our candidates and improve the overall quality of our hires.

Hiring is important because people are important

Hiring is one of those things where if you do it right, many mistakes can be overcome, and if you do it wrong, the best strategies and tactics will not help you.

What I described is our process, at Codecademy, that is based on our values and constraints. There is multiple ways to try to solve this problem. Some companies prefer leveraging AI and automated, timed challenges. Others will actually have a candidate shadow an engineer for the day.

Whatever you decide to do, I challenge you to do it thoughtfully. Your company is its people, and the hiring process is the beginning of each of those relationships.