SOC2 - GN-87 Website Performance and availability monitoring

[JenRed777]

Due: February 15, 2022

Sourcegraph uses a website performance and availability monitoring tool to help ensure awareness of web application uptime. Uptime alerts are sent to the Security team and issues are logged and tracked through remediation.

Notes: Use GCP Uptime check, sends alerts to a slack channel, security team monitors this channel. Should provide links to the Terraform code that does this.

[filiphaftek]

1. Uptime check from prod cluster (actual state):

   • blakbox_exporter is used to scrape internal and external Sg cloud endpoints
   • static http checks are defined for internal and external endpoints
   • alerts thresholds from static http checks are defined
   • alerts are sent via Opsegnie and to Slack channel #opsgenie
   • indcident.io is used to track and fix issues

2. Performance monitoring on production cluster (actual state):

   • custom metrics for performance are defined and alerted
   • custom page speed rules are defined and alerted
   • alerts are sent via Opsegnie and to Slack channel #opsgenie
   • indcident.io is used to track and fix issues

3. TODO

   • add GCP uptime checks with alerts - PR

[filiphaftek]

@JenRed777 created document

[JenRed777]

Need to add the Delivery Portion to this document

[daxmc99]

@JenRed777 Updated the doc with steps for managed instances

[JenRed777]

This looks good and has been updated on the spread sheet

[RafLeszczynski]

dependent on the https://github.com/sourcegraph/infrastructure/pull/3046

[daxmc99]

Merged https://github.com/sourcegraph/infrastructure/pull/3046 This codified the change we made in the above document

[JenRed777]

This one is complete