arachnys / cabot

Self-hosted, easily-deployable monitoring and alerts service - like a lightweight PagerDuty
MIT License
5.59k stars 593 forks source link

Any API Issue with HostedGraphite Triggers Cascading Alerts #668

Open jjafuller opened 5 years ago

jjafuller commented 5 years ago

Our graphite provider occasionally has issues where we get failed API calls when pulling for metrics. These failures tend to result in nearly all the alerts triggering. Is there a way to have Cabot ignore graphite API failures and just send a single notification so that we know something happened without having to triage hundreds of alerts?

MrSaints commented 5 years ago

Related #596