OOB Deploy Request

[oleksii-morgun]

Instructions

Please fill out the necessary details and list the PRs related to the OOB deployment in the sections below.

PRs Related to OOB

• Please provide the list of PRs related to the need for this OOB Deploy below:

  https://github.com/department-of-veterans-affairs/vsp-infra-application-manifests/pull/1901 https://github.com/department-of-veterans-affairs/devops/pull/12994

Active Daily Users Impacted

• How many active users are impacted at this time and day.

  We are seeing an average of about 5 requests per second over the last 24 hours so around 400k requests failed due to this issue.

  Has fix been confirmed in Staging?

• [x] Yes
• [ ] No

Description

• Please provide the details/reason for the OOB Deploy

  Reverting environment configuration deployed on 5/8/2023 that resulted in MHV API services become unaccessible

Verify The following

• [x] The OOB Deploy Request is after the 2pm EST cutoff for regular deploy.[^1]
• [x] The OOB Deploy Request is critical and must be resolved before the next automated deploy.
• [x] You are prepared to create an Incident Post Mortem^2 within two business days.

[^1]: (See Deployment Policy and Deployment Schedules

Performed by Platform Support Team

• [ ] PagerDuty OOB Deploy Incident Opened
• [ ] OCTO-DE staff acknowledgment of Request, via /pd trigger
• [ ] Notification is posted in the appropriate Slack support and team Channels
• [ ] Infrastructure/Operations has acknowledge the Requests

• [ ] Security Team has Reviewed the requests

  CC: @department-of-veterans-affairs/vsp-operations , @department-of-veterans-affairs/vsp-product-support

[BillChapmanUSDS]

OOB approved

[va-vsp-bot]

This issue is stale. If it is no longer valid, please close issue. Otherwise please update