On the GDS SWE standardised interview format proposal

[ckkok]

Calling for feedback here...

[sohymg]

Feedback from ESTL

For context, this is ESTL's current SWE interview process

Take-home coding challenge

• Main goal: serves as a quick screening/filtering mechanism
• Allows the candidate to work on a “real-problem” without interview pressure
• Solutions to our take home should not be Chat-GPT-able
• Most importantly, solutions can be tested for correctness in a fully automated fashion
• Should not generally take more than 1 hour to solve
• Candidate given a few days for submission

Feedback: we moved away from the CRUD app as take-home because it takes too much time for the candidate + it makes it very difficult for automated evaluation as the key in a CRUD app is the code design which requires human evaluation. The purpose of the first stage is to help out quickly filter out candidates who cannot code hence, we are not keen to have a separate run-through session with all candidates.

Live coding

• Main goal: to ensure candidate really can code and not because he/she received help with the take-home coding challenging
• Live coding will be JS based
• Simple coding question. We strive not to ask “puzzle” style questions where you need knowledge of esoteric data structures or clever ideas to solve (i.e. some leet code style questions).
• Candidates will be assessed on 5 metrics:
  • Reading Code
  • Writing Code
  • Problem Solving
  • Testing
  • Communication
• Each of these metrics will be graded on a 0-2 point scale, with 1 point being Meets Expectation and 2 points being Exceeds Expectations.
  • It is possible to get 0.5 points or 1.5 points if the candidate performs in between Below and Meets, or Meets and Exceeds.
• Live coding questions will have the question already written out, as well as boilerplate code that provides a simple framework for running tests (together with one test case).
• Candidate's ability to understand the boilerplate code is part of the Reading Code assessment

System design

• Main goal: to ensure the candidate can come up with a good design in a natural setting (take their time at home) and ability to communicate their design clearly and defend their design choices
• Questions are take from real-life features in our day-to-day work.
• Candidates are given up to a week to prepare presentation materials

Feedback: People don't come up with system design on the spot in our day-to-day so we want to mimic reality and give them time to come up with the design. At the same time, we can assess their presentation and communication skills which are important for more senior positions.

[ckkok]

Collating feedback from various sources here

From Calvin:

1. If round 1 for pair programming is to build new feature for take home challenge only, timing is good, but probably budget for 1.5 hours.
2. I still like to have round 2 for everyone including juniors, but you may decide to reduce interview effort. Either that, or a reduced 30-min design question.
3. Mettl is used as a pre-screening tool for candidate's basic technical knowledge.

[ckkok]

Collating feedback from various sources here

From Selwyn:

1. Is Stage 0 meant to be a filter? After the initial CV screening, if the take-home assessment is too simple, then the assessment is too simple/straightforward, then most/if not all candidates will just pass that round. In that case, the effectiveness of stage 0 doesn't really match our investment in terms of effort to assess the candidate. Even if we can do full automation, there are plenty of tutorials on how to do a CRUD, so it may end up being how well the candidate can follow instructions. Something more unique and targeted may thus work better unless our objective for stage 0 is really to filter those that are totally not technically inclined.
2. Regarding stage 1, if it's just a simple CRUD, there isn't too much to walk through. A walkthrough may be more suited for a system design question. We tried sending a take-home system design question to candidates, followed by a session to walkthrough the proposed implementation + discussion on design decisions. That worked fairly well, but should be limited to senior candidates.
3. I noticed that there is no resume deep-dive. (Or is this done as part of stage 3?) In a previous discussion with Benny, our conclusion is that the SWE competencies should be assessed based on what the candidate has done and not what the candidate is able to show (which can be prepared actually, given the sheer number of resources out there to prepare for system design questions) If we are looking for lvl 3 competencies, I think digging deep into the resume is inevitable. The actual system design/take home etc is more for us to verify the capabilities of the candidate.