elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.8k stars 8.19k forks source link

[Alerts] Create alert rule comparing data to the previous day / week #173716

Open ruflin opened 10 months ago

ruflin commented 10 months ago

Currently when creating a for example a log threshold rule the user must set a fix condition like "above 75 errors for the last 5 minutes". In some scenarios, instead of having a fixed rate, it would be more useful to compare the data to the previous day during the same period. For example: "log rate should not be > 5% more then during the last day for the same period". Same could be done comparing it for example to the last week to take into account different traffic during weekdays and weekends.

elasticmachine commented 10 months ago

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

paulb-elastic commented 10 months ago

@vinaychandrasekhar this is a good example of comparing based on seasonality.

There are a number of considerations here, such as, where this should go - is the log threshold rule the right place, or should it be the Custom Threshold rule? Or would it be more appropriate being in a dedicated rule for comparing signals against other time periods?

It would make sense for this to be represented in a project, to look into the best approach that fits in with more than just logs, and considers different use cases.

jasonrhodes commented 5 months ago

This sounds to me like a riff on the "Comparison" feature available in APM for some time, right? See screenshot below:

Image

I just spoke to @vinaychandrasekhar about this and his suggestion was to create an issue to plan how to introduce this same functionality not just for alerting rules, but for observability as a whole (charts, graphs, embeddables, alerting rules, SLO graphs, etc etc) and make sure we have a plan that works as widely as possible, then revisit implementation in the rule space. @alex-fedotyev is this something on your radar at all, in a broad sense?