department-of-veterans-affairs / abd-vro

To get Veterans benefits in minutes, VRO software uses health evidence data to help fast track disability claims.
Other
18 stars 6 forks source link

Tech Spec: Create an oncall rotation and process for engineers (SecRel/Dependabot) #2435

Closed agile-josiah closed 5 months ago

agile-josiah commented 6 months ago

VRO engineers have to currently volunteer to have an informal process for being monitors for SecRel and Dependabot. Also individual engineers are siloed in some instances for a POC between partner teams. Formalizing this process will allow people to fairly distribute the work and prevent a single point of failure and knowledge on the team. This will also allow our partner teams to feel more supported when reaching out because instead of a person who may not be available there is a role that can help or be a liaison for help.

Acceptance Criteria

  1. There is an oncall rotation process that distributes a primary/secondary fairly and weekly.
  2. All engineers have an agreed place to put their availability.
  3. A slack bot reminds the team of the oncall schedule, as well as an @vro-oncall role in slack that will ping the current primary and secondary as a POC for any stakeholders and partner teams.
  4. Documentation is added for how the oncall rotation works, what we support, and playbooks for oncallers to address issues that may come up and POCs that can help unblock vro during failure states.
  5. A method for engineers to swap oncall weeks in cases of conflicts or emergencies.

Notes about work

agile-josiah commented 6 months ago

https://sre.google/sre-book/being-on-call/

paulsackley-nava commented 6 months ago

@agile-josiah to do a tech spec for this sprint