3/10 Prod Deployment Failure

[gracekretschmer-metrostar]

User Story or Problem Statement

As a CMS tech lead, how may I most strategically prevent the prod deployment failure from happening in the future?

Identify how staging handles clearing drupal's cache The rollback procedure is a manual one, it would be nice to have something more automated

Description or Additional Context

On 3/10 the deployment to prod failed. After investigation, the CMS team discovered that a dependabot bump caused the prod deployment failure. After reversing the dependabot bump, the team was able to resolve the error and prod successfully deployed.

Relevant Links

• https://dsva.slack.com/archives/CT4GZBM8F/p1710186762879049

Hypothesis

The team hypothesize two following solutions to prevent this situation in the future:

• Identify how staging handles clearing drupal's cache
• The rollback procedure is a manual one, it would be nice to have something more automated

In addition, if we could broaden our testing then we could catch these issues earlier in the pipeline.

Acceptance Criteria

• [ ] The root cause of the failure is understood.
• [ ] An understanding to why the failure was not caught in a lower environment.
• [ ] A plan of action is determined to implement solutions that would help prevent the failure from happening in the future.

Team

Please check the team(s) that will do this work.

• [x] CMS Team
• [ ] Public Websites
• [ ] Facilities
• [ ] Accelerated Publishing

[7hunderbird]

@gracekretschmer-metrostar I'm just taking a fresh look at tickets that I'm assigned.

When I look at this ticket I can use the slack link to research what happened, but the experience of working on it is a bit stale because it's been a while.

I guess I wanted to get it back on your radar and perhaps work together on next steps.