Interview Process

[chawang90]

Hey folks so I've been thinking a bunch about our interview process and think this is somewhere we should think from scratch, even if end results are similar.

Here is the framework I have when I think interview process:

Tools. Streak for pipeline management & attachment/resume collection, Slack for team-wide updates, templates for actual interviews and note collection. Funnel I like the idea of 3 steps but regardless of how many steps, each step should significantly screen the candidate pool, and increase in time commitment. Lets us be smart with our time. Technical vs Non-technical Happy to have slightly different processes for these, although neater if we can keep them similar.

Here's what's in my head right now: (read the baseline hiring process first)

• One person on each team should be responsible for each role (role owner), and every application should have one point person who is responsible for pushing that person through the funnel from application through first day of work (application lead).
• For inbound applications role owner should file inbound applications into Streak and seeing if there are any deal breakers that send them right to pass (we should have an template email for hard passes). If they go forward, role owner can delegate applications to different leads.
• For referrals the referrer should be the application lead.
• We should have a template for each interview and notes.
• So far for the first 30 min interview I've done:
  • 10 min explaining the job: History of the need, why we structured it the way we did, what we're hoping to see in a qualified candidate (ie. excellence in these specific areas)
  • Then 20 minutes of asking them questions:
  • Relevant Experience, ie. Here's what we think this job needs, tell us how you match up to these buckets of experience
  • Priorities, ie. what's most important to you in looking at opportunities (Team, Type of Work, Mission, Lifestyle, Growth Opportunity, etc)
  • Logistics, ie. when could you start and are there any deal breakers or things you need that we should know about off the bat.
  • Q&A, ie any other questions.
  • My notes for the interviews follows this format (Experience, Priorities, Logistics) + My Thoughts and Next Steps
• Notes currently go into Streak and in the #hiring channel, not sure if that's the best way to do it, especially as we move forward beyond first interviews.
• Love the idea of 1 on 1s for last half of team fit day, think it would be cool to have a "question bank" so that people can not ask exactly the same questions to candidates over and over again. If we all ask questions about different areas that we are individually interested in, they'll get a better understanding of what we're like and we'll get a much broader understanding of them as well.
• I think we need to make sure these interviews are difficult. It behooves us to make people earn their offers and to show candidates that are more senior or more decorated than we are that seniority doesn't mean much in an arena that hasn't been explored before. Also I want for there to be opportunities for us to see how candidates ask for help and struggle, since those things are important for team dynamics.
• I would love to use actual decisions we've had to make/problems we've had to solve in the past as exercises for employees. A perfect example of this would be the "how do we reopen the marketplace after shutting it down to private" situation. Split opinions, all with merit, but I want to see how, not what.

ThoughtS!?

[chawang90]

This is probably clearer:

Process:

• Informal Meeting
  • 10 min pitch
  • history of role
  • problems to be solved
  • skill sets needed
  • 20 min interview
  • skill set specific experience
  • priorities in job hunt
  • logistics & deal breakers
  • questions & answers
• Skills Assessment
  • 45 minute exercise
  • 15 min Q&A
• Culture Fit
  • 1-2 hour deeper dive exercise
  • To see how they ask for help & struggle
  • Doesn’t have to be “solvable"
  • Team Lunch
  • Half day of 1 on 1s
  • questions from the question bank!

Organization:

• Role Owner
  • Writes Job Listing
  • In charge of outbound marketing and overall management of role
  • Default application lead for all inbound applications unless delegated
• Application Lead
  • Either direct connection to application or assigned by role owner
  • Responsible for updating status of application and pushing through funnel, from application or assignment through pass or first day of work.

Communication:

• All leads stored in Streak. Role Owner responsible for keeping streak updated for those roles
• Application Leads should update individual application's status after each step
• Notes for each interview are posted in hiring within two days of interview as well as appended in Streak.
• Maybe there's better software out there for this process.

[talsafran]

Every candidate should go through three interviews that help us find the right person for the job.

Before they are interviewed, the candidate should show some relevant work that will help give us an initial screen and something to talk about in the first coffee interview. They should also get a blurb about Josephine, the position, and our values so that they can self-screen if our culture doesn’t match the kind of company they want to work for. Values in particular should be emphasized here so that they understand how important they are to the Josephine team and make sure they align.

Chill. We might want to write a step 0 about screening, with some guidelines on what the initial screen looks like –– ie, developers must send code samples, cook ops send a writing sample, designers send a portfolio, etc.

A resume or online profile can tell us a lot about what a candidate has done, but the first thing we need to find out is what they want to do next, and whether that matches what we’re looking for at Josephine. This interview can happen over the phone, Skype, in the office, or out at a coffee shop.

The idea is to spend 30 minutes getting to know the candidate and letting them ask questions about Josephine and the position. This has screened about 80% of candidates so far, which seems about right. It should be pretty transparent if the candidate’s experience or interests are misaligned for the job.

This interview should also focus on selling Josephine and recruiting promising candidates, as this is their first real contact with Josephine and their desire to work here will be formed in this meeting.

So just to be clear, the goal here is to see first (1) what motivates the candidate for the job and, secondarily (2) sell the candidate on Josephine as a company.

Regardless, it seems like the focus here is more "how jazzed they are about Josephine" (or why) and not necessarily at least "how jazzed they are about doing Software Engineering at Josephine." Basically, what I'm getting at is –– if that's the case then we should be extra tight with our skill screening. Otherwise we may end the meeting with "well, I like them, but I still don't know if they're good enough at their job to merit 2 hours of investment."

The technical interview is a chance to dive into the candidate’s ability at the job they’re interviewing for. The lead on this hire should prepare two exercises which will each take 45 minutes, one which goes into high level thinking for the position and the other which focuses on more low level implementation.

The last 30 minutes should be used for questions and answers from both sides. This interview should filter out 50-70% of candidates (of the remaining 20%), and by the end of it, the lead interviewer should have an opinion on whether or not the candidate is the right person for the job.

I think two hours is right, but I'm still thinking of The Right Technical Challenge(s) here, and that might dictate whether we need more or less time.

A thought –– I've been thinking of having some sort of homework assignment as a pre-screen. That way we'll have a standard screening mechanism that we can send as opposed to "perform", which takes work off our plates. Then in the 2 hour interview, assuming they pass the screen, we can look at their project and (again) have a standard followup interview.

Thoughts? Regardless, I'll need to think more about appropriate challenges – @joyjding @zeke anything you've seen that you like?

Finally, the candidate should come in for a whole day of talking to and working with the team. In the morning they'll drill into a much more challenging problem than they had to deal with in their technical interview. We want to get a sense for how candidates handle something out of their comfort zone, how they ask for help, etc. It’s also important that candidates feel like they’ve earned their offer by the time they get it, so this part of the process should be hard!

After the morning exercises, candidates go out to lunch with the team before coming back to the office for one on ones with everyone. As we get a little bigger, this might need to be one on twos to fit conversations into a reasonable amount of time.

My guess is that the best sequencing for this is:

1. Skill interview followup (harder than before, and more personalized since we know them now)
2. Lunch with the team
3. One on ones

If for whatever reason the candidate doesn't pass (1), then I think it's okay to end the day early to respect both sides' time.

[talsafran]

More thoughts on the technical interview process.

Basically, I'm alright with this three step format, but would like to cover two "unofficial" steps that I don't think we should necessarily publicize (but keep in mind with all candidates)

0. Screening

What are the criteria we screen for?

• Minimum 2 years experience (we post 3+ on the listing but 2 should be enough)
• Experience with web frameworks.
• Good projects to look at (this will vary in practice –– see step 1')
• A good cover letter is a big bonus.

This should help us weed out folks that very obviously don't meet the skill requirements.

If all things are good, we move on to step 1 to get coffee with the candidate.

1a – Take Home Exercise

In some cases, it's hard to judge whether a candidate meets our skill requirements just by reading a resume, and we want to avoid committing two hours in order to figure that out. You probably meet the requirements if you have good open source projects or wrote Ruby at Airbnb, and probably don't if you just graduated a bootcamp or don't have any meaty projects online.

But what if you worked at a lesser known company for a few years, and don't have very obvious projects online? That's why we should do a take home exercise. This comes after the coffee meeting for two reasons:

1. We're still respecting the candidate's time, they'll be more likely to agree to a coding exercise after we've met and gotten them jazzed about Josephine. Brian Mason and Ben Miller will probably get the coding exercise.
2. The coffee meeting is only 30 minutes as opposed to two hours, and it still gives us a sense of whether or not this is a person we'd want to spend time with.

For some cases I still think it makes more sense to send the take home exercise before coffee. An example is someone we reject (like Claire Schlessinger, a bootcamp grad) but has asked us anyway for a coding challenge. That way we don't have to commit to any time.

The Exercise

I've started to talk with @joyjding and @zeke about what exactly the exercise will look like. Starting to get a good sense of the requirements though:

• Takes two hours at most. If it's easy for you, less than an hour.
• Easy to set up, with good instructions
• Framework agnostic. You don't have to know Rails, React, etc.
• We'll start with Ruby, but if enough people ask we can write another version in Javascript.
• Some sort of "acceptance test" the candidate can run to know when they're "done"

[talsafran]

Oh. Last thought I had about hiring.

References

How do we feel about asking for 2 referrals from each candidate, if we don't have any mutual connections already?

Keep it short, maybe just two questions:

• What is this candidate's biggest strength?
• What is this candidate's biggest weakness?

I think these can better inform the way we approach the candidate, especially in the last stage when we do 1-1s. So maybe have this be between steps 2 and 3?

[joyjding]

This might not need saying, but I think that in all of our interviews, we need to fully present, engaged, and friendly. Even if a candidate doesn't work out for us, we want the candidate to feel respected and impressed by the end of the interview. Candidates in fields talk to other potential candidates, and I'd love the person's takeaway to be: I had a great time meeting the people at Josephine, and the interview was challenging but fair. I wasn't the right fit, but I'd highly recommend it as a place to consider working.

[sike13]

So @talsafran's comments were pretty specific about screening for product team but I absolutely agree that in addition to creating the job listing, the role owner should be responsible for taking a 1st pass at outlining how screening and the interview process work. For example screening for product vs screening for the cook team will look very different — the handbook doesn't need to dictate what the screening process is, but rather dictate that the screening process needs to be thought about and outlined in greater depth (ideally before hiring begins).

[egustafson5]

@chawang90 I like this process and thinking, overall. After having done two 30 min chats, I think it's good to end the 30 min chat (which is really 45+ min) knowing:

1. Why this person is jazzed about Josephine
2. Why they're jazzed about this role & how they view their relevant experience (corresponding to specific buckets of work we outline)
3. Where they've come from and where they're going (personally + professionally), and a better idea of Josephine/this role makes sense for them at this time
4. A better idea about how they think and what their strengths, weaknesses, and interests/passions are, so we can assess how they fit in with the rest of the team and fill the gaps we have — e.g. I ask about what different organizational or PM tools they use, what skills they're hoping to focus on and develop personally/professionally, etc.

We should also figure out how we're comparing candidates. I found myself already giving candidates scores based on experience/skill in each of the buckets of work, as well as team fit. Do we want to develop a rubric? Is this by team? Do we move all qualified candidates forward, or do we cap it at top 3 make it to round 2?

@talsafran I'd like to end the 30 min interview having assessed them in the context of the particular role, not just how excited they are about Josephine in general. Also, your screening steps make sense.

@sike13 Yeah, could be Step 0 or 1a, but before the 30 min interview we should absolutely see samples of writing (something relevant, not academic) and social media work.

@joyjding I totally agree — my positive and negative experiences applying and interviewing different places have definitely affected how I feel about those companies, and people definitely talk. Also, looking for a job is rough, and we should be kind to everyone :)

[chawang90]

Lots of great comments here.

Agree with @sike13 that screening should be mentioned in the handbook as a role-owner and role-specific responsibility.

@joyjding I think that's very important to bring up, actually. Something that seems obvious but that we should always be reminding ourselves of, especially when we're busy.

@egustafson5 do you think those questions are a better way to frame the 30 min interview than the original outline? Happy to put it in that format, in some ways seems more straightforward

@talsafran would 1a. be an optional step? what percentage of applications are you hoping you'll need to engage in 1a. before bringing them in for 30 minutes? (or is it after 30 minutes?)

[chawang90]

@MattJorgensen @terrybetts24 looking for your input here.

[joyjding]

This is a helpful framework for thinking about hiring at a high level

Almost all hiring questions fall into one of three categories: 1) Can the person do the job? (this is usually covered by work experience, portfolio, skills assessment)

2) Will the person do the job? (Motivation, what will keep them going when it's hard/difficult? What are they excited about in the position? Where are they hoping to grow, and does that align)

3) Culture fit (Collaboration, Communication, interactions with other team members, will they strengthen the culture and values)

[chawang90]

Latest for review here

[MattJorgensen]

Looks great to me.

Two thoughts:

1.) Who do we want taking the informal/coffee chats? After talking to two or three people that were direct intros to me (through friends), but that weren't for business team roles, I personally think even these informal chats should be taken by the application leads. (still great to chat with friends casually about josephine, just feels like not worth work hours)

2.) Who do we want to come into the full-team fit day? I feel like we should only bring two people to this stage if we are really on the fence between two candidates. Otherwise, we should take a sequential approach by bringing in our leading candidate first, and only bringing in an additional if we rule them out, or feel 'meh' after the fit.

[chawang90]

@MattJorgensen can add clarification for 1., feel like 2. can be left for teams to figure out. I agree with your logic but it might not be necessary to publish for all candidates to see.

[terrybetts24]

Very thorough, nothing to add.