search

OpenSLO / slogen

tool to create and manage content for reliability tracking from logs/event data.
Apache License 2.0
78 stars 6 forks source link

Missing Data Alerts #54

Open lswith opened 2 years ago

lswith commented 2 years ago

The Monitors created don't have useful missing data alerts.

I'd like to be alerted if I screw up a query and the total count goes to 0 for x minutes.

agaurav commented 2 years ago

I think it should be achieved by adding MissingData to triggerFor conditions but let me verify this with some actual alerts.

agaurav commented 2 years ago

This seems to be non-trivial as the scheduled views that generate the SLI data from raw logs uses the fillmissing operator to generator the SLI values even if there were no log lines.

One way to solve this will be to add an option in the spec to create MissingData monitor on the raw log query itself. I will try to roll it out by the end of week.