Content Release Daily Process Fails when GitHub Runners are Unavailable - Githubissues

department-of-veterans-affairs / va.gov-cms

Editor-centered management for Veteran-centered content.

https://prod.cms.va.gov

GNU General Public License v2.0

97 stars 69 forks source link

Content Release Daily Process Fails when GitHub Runners are Unavailable #18923

Closed 7hunderbird closed 1 month ago

7hunderbird commented 1 month ago

User Story or Problem Statement

This morning CMS support noticed that Content Release did not start
DevOps investigated and saw that there were no GitHub Runners available
There were no GitHub Runners because the Auto Scaling Group had failed to refresh them
It failed to refresh because an AMI ami-0cc4f1573df8d60ce got pruned

CleanShot 2024-08-13 at 10 27 16

Description or Additional Context

Run Build Platform Base AMI (Manual)
Run Build GHAR
Run an Instance Refresh on the dsva-vagov-content-build-gha-runners-asg

CleanShot 2024-08-13 at 10 28 33

Acceptance Criteria

[X] Complete a content-release
[x] Add doc to confluence on steps

7hunderbird commented 1 month ago

Created https://vfs.atlassian.net/wiki/spaces/PCMS/pages/3378380804/Refresh+GitHub+Runners+for+Content-Release

edmund-dunn commented 1 month ago

@7hunderbird You captured everything nicely! I can't see anything you missed!