Add detection of flapping alerts

canonical / grafana-agent-operator

https://charmhub.io/grafana-agent

Apache License 2.0

4 stars 10 forks source link

Closed err404r closed 3 weeks ago

err404r commented 3 months ago

It would by nice to have a separate rule to alert when we have a flapping alerts

err404r commented 3 months ago

Something like changes(ALERTS_FOR_STATE{severity = "warning"}[1h]) > 5 It's mostly important for warnings, because they could be missed

err404r commented 3 weeks ago

Inhibition alert are not working that way, so we can't use flapping alert to properly suppress flapping alerts