search

gitpod-io / gitpod

The developer platform for on-demand cloud development environments to create software faster and more securely.
https://www.gitpod.io
GNU Affero General Public License v3.0
12.79k stars 1.23k forks source link

Change GitpodWorkspaceTooManyRegularNotActive to an alert and update criteria #8051

Closed kylos101 closed 2 years ago

kylos101 commented 2 years ago

Is your feature request related to a problem? Please describe

The current warning goes unnoticed (as it is a warning and does not page), and fires too frequently.

Describe the behaviour you'd like

Change the warning to an alert (which pages), and alter the criteria so it triggers well, and create a related runbook.

Describe alternatives you've considered

This seems like it might work well rate(gitpod_ws_manager_workspace_activity_total{active="false"}[1h]) > .3

Additional context

Look at the last 14 days in Grafana to get a feel for what that might look like.

princerachit commented 2 years ago

We had a spike today which was followed by stuck in stopping state image