Update instructions for using StatusPage

ccostino commented 4 years ago

In order to better serve our customers, we need to make sure our use of StatusPage results in messages being sent out in a timely matter.

We recently ran into a situation where we cancelled a scheduled maintenance activity, and no cancellation notices were sent out. Furthermore, the status wasn't updated on the cloud.gov StatusPage either to reflect the current state of the services; a customer pointed out that those maintenance notices were still visible.

Acceptance Criteria

[ ] GIVEN a need to use StatusPage to update or cancel a scheduled maintenance, a platform operator can double check that they are following the right steps to ensure that customers are receiving the proper notifications in a timely manner on the Making announcements page.

Security considerations

None; the platform operator manual for maintenance notices is already public and contains instructions on the steps to take in StatusPage to manage maintenance notices. This would be an expansion of those steps.

Implementation sketch

Please see our notes on the incident for full details. What this all boils down to is making sure we have clear instructions on the following:

[ ] What steps to take when updating/cancelling a scheduled maintenance in StatusPage.io
[ ] Confirming that notifications are sent out to customers
[ ] Confirming that maintenance notices are updated to accurately reflect the state of the system and the work being performed

onelittlebecca commented 4 years ago

Something similar to this?

pburkholder commented 4 years ago

There was an effort in TTS to consolidate our IR processes along with the TTS tech portfolio team and the other teams that have to have one for their ATOs. It was going well until two key drivers left (Marshall and Hillary) and we've not picked it up again.

Hillary made a great draft flowchart here: https://app.mural.co/t/gsa6/m/gsa6/1578454101053/2be4f2baba395ba881353094f668cd4ef22559b8

Anyhow, this is clearly a pain point so we should get a card on the board to tackle our guidance. I agree though that the statuspage process is better as as public doc than an internal runbook to keep our process consistent (and to help others).

On Tue, Jun 2, 2020 at 9:24 AM Becca notifications@github.com wrote:

Something similar to this https://cloud.gov/docs/ops/security-ir-checklist/?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cloud-gov/product/issues/1385#issuecomment-637539954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJHWCUW4SIYZLYPIYYPAK3RUT4Q5ANCNFSM4NQEYEXQ .

--

Peter Burkholder cloud.gov compliance & security | co-lead DevOps Community of Practice https://digital.gov/communities/devops/

Solutions https://www.gsa.gov/about-us/organization/federal-acquisition-service/technology-transformation-services/tts-solutions | Technology Transformation Services https://join.tts.gsa.gov/tts-offices/

|

GSA http://www.gsa.gov/portal/category/100000

202-709-2028 <(202)%20209-2028> | peter.burkholder@gsa.gov peter.burkholder@gsa.gov

| pronouns he-him https://www.mypronouns.org/he-him Free/Busy Calendar https://calendar.google.com/calendar/embed?src=peter.burkholder@gsa.gov

ccostino commented 4 years ago

Yes, something similar to that @onelittlebecca, that can walk someone through the process and provide a quick little checklist to make sure nothing is missed!

@pburkholder, the intent is for this to be a public doc, not an internal runbook. Sorry, I may have misused the terminology here, but I'll make this clearer above!

cloud-gov / product