department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

Launch Prep Tasks #42805

Closed pjhill closed 2 years ago

pjhill commented 2 years ago

Description

In order to be able to launch synthetic monitors we need to finish a few tasks first. Let's complete the below tasks and launch!

Here're the Go / No Go notes: https://vfs.atlassian.net/wiki/spaces/AP/pages/2198700033/5+17+22+Office+Hours+Agenda+and+Notes

Tasks

Acceptance Criteria

rbeckwith-oddball commented 2 years ago

The second item: "Verify whether free account is blocking our launch"

rjohnson2011 commented 2 years ago

I would also add 'Fix alerting based on browser changes' since this is now causing more triggers than desired.

pjhill commented 2 years ago

Right, I guess all that was wrapped up in that one little statement. To unpack it a bit more --

  1. What are the implications of a single browser being monitored instead of 3?
  2. Is there a fundamental problem with only verifying a single browser at a time?
  3. What rule changes for alerting are necessary to keep the spirit of the monitoring/alerting?
pjhill commented 2 years ago

I've added the detailed action items list to the ticket summary.

rjohnson2011 commented 2 years ago

I'll look into what effect the browser changes have made on the alerting. I can see for `VA.gov - login' which was failing this morning, was due to having 0 retries enabled. I changed the default retries to 2 for all the Managed by Terraform synthetic monitors, to cut down on false alerting, and go in sync with our initial approach to require 2 / 3 failures before triggering an alert.

pjhill commented 2 years ago

These tasks have all been completed.