Open ruflin opened 11 months ago
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)
@vinaychandrasekhar this is a good example of comparing based on seasonality.
There are a number of considerations here, such as, where this should go - is the log threshold rule the right place, or should it be the Custom Threshold rule? Or would it be more appropriate being in a dedicated rule for comparing signals against other time periods?
It would make sense for this to be represented in a project, to look into the best approach that fits in with more than just logs, and considers different use cases.
This sounds to me like a riff on the "Comparison" feature available in APM for some time, right? See screenshot below:
I just spoke to @vinaychandrasekhar about this and his suggestion was to create an issue to plan how to introduce this same functionality not just for alerting rules, but for observability as a whole (charts, graphs, embeddables, alerting rules, SLO graphs, etc etc) and make sure we have a plan that works as widely as possible, then revisit implementation in the rule space. @alex-fedotyev is this something on your radar at all, in a broad sense?
Currently when creating a for example a log threshold rule the user must set a fix condition like "above 75 errors for the last 5 minutes". In some scenarios, instead of having a fixed rate, it would be more useful to compare the data to the previous day during the same period. For example: "log rate should not be > 5% more then during the last day for the same period". Same could be done comparing it for example to the last week to take into account different traffic during weekdays and weekends.