Partner Team Production Deployment Workflow - Prototype, Soft Launch, and Comms

[nelsestu]

Description

This issue is a continuation of #2795, and is based on findings from #2672. At this point we've designed, documented, and prototyped the scheduled deployment process in Slack workflows.

Why

To ensure recommendations made are feasible for implementation and mitigate existing partner team pain points and gaps highlighted in the research phase

AC

Please see #2795 for the other half of the overall effort to bring about scheduled deployments

• [x] Test workflow internally on mock channels for demo purposes: #benefits-vro-support and #benefits-vro-on-call
• [x] Duplicate workflows to real channels for testing purposes
• [x] Complete migration of deployment strategies by rolling over to the new process
• [x] Draft and send official comms message to #benefits-vro-support channel

• [x] Notes

• This ticket is referencing deployment requests partner teams make to higher-level environments.
• Replacement for #2795
• Reference work from #3042

Outstanding items:

• If a partner team engineer's confidence is low, how do we want to engage with them for additional testing?
  • if still at Low Confidence by Thursday, detour to Opt-out workflow
  • Follow-up: Ask/remind partner teams
• Do we want to provide guidance on what testing they should conduct before scheduled deployment? and to which environment?
• What happens if deployment is unsuccessful (i.e. try to deploy on Thursdays, encounter issues, rollback)?
  • Follow-up: communication to partner teams

Resources

Mural link GitHub & Slack integration features

[BerniXiongA6]

Per @bianca-rivera -- this ticket should carry over to a future sprint since it wasn't worked on in Sprint A or B. I'll untag the ticket from Sprint A and notate this on the current Sprint epic.

[bianca-rivera]

Following the steps taken with the Incident Report workflow, I updated that documentation of Deployment workflow should be added to the Deployment Policy wiki page

[BerniXiongA6]

@BerniXiongA6 -- will roll over to Sprint 1

[bianca-rivera]

Expansion of action items based on 16th min notes:

• [ ] Update Deployment policy wiki page:

  • [x] incorporate latest decisions from 16th min notes doc ( @brostk )
  • [ ] delineate VRO vs partner team responsibilities/expectations for process (tuesday-thursday) ( @bianca-rivera )
  • [x] process flow diagram ( @brostk )

• [ ] Update Opt-in/Opt-Out Workflows: ( @bianca-rivera )

  • [x] integrate opt-in and opt-out via link (TBD whether possible)
  • [x] add comms touchpoints: 1) get validation from partner teams 2) sign off from VRO
  • [ ] Determine alerts/notifications:
  • [ ] determine whether subscribing #-alerts channel to abd-vro repo make sense/works --- share updates with VRO team, get feedback ---
  • [ ] come up with and present options of varying levels of notifications to partner teams during Partner Team Time (Thursday) meeting
  • [ ] Get feedback from partner teams:
  • [ ] what type of notifications do partner teams want/need?
  • [ ] cadence/frequency?
  • [ ] any other type of notification?