Closed dginther closed 3 years ago
Need AWs access before taking this one.
Might an equal & opposite outage occur if a lookup was versioned? In such a scenario, the developer updates the value in credstash but doesn't bother to change the version in the lookup. They expect the new version to be used, but instead get the old one.
Both scenarios hinder on the fact that the developer made an assumption about how the lookup occurred rather than specifically verifying it.
"I updated the value in credstash, therefore I assume that..."
I'm not sure one assumption is any more valid than the other. FWIW, I would tend to assume the latter 🤷.
Removed cms related items from changes due to cms-infra team member request at this time (ref: https://dsva.slack.com/archives/CJYRZK2HH/p1609781236065600).
Can we close this since we are moving to parameter store now? Not sure how much more time we want to invest in updating credstash versions vs changing to parameter store calls which is arguably out of scope and another ticket.
Closing, moving to Parameter store.
Description
What details are necessary for understanding the specific work or request tracked by this issue? There was an outage caused by an un-versioned credstash lookup, when vets-api deployed, it pulled the latest evss prod key, which had been placed in credstash in preparation for the new key to be deployed. Since the new key had a higher version number than the old key, the new deploy pulled the new key, even though it was not ready to be deployed yet.
Background/context/resources
Any additional context for the reader to know, if applicable https://dsva.slack.com/archives/C30LCU8S3/p1598042685002000
Technical notes
Notes around work that is happening, if applicable
Tasks
Definition of Done
Reminders