search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.69k stars 8.12k forks source link

[SLO] Burn rate alerting - Better UX for burn rate rule updates based on SLO duration updates #179941

Open dominiqueclarke opened 5 months ago

dominiqueclarke commented 5 months ago

Describe the feature:

The SLO duration (7 days, 30 days, 90 days), is critical to determining the default burn rate thresholds. Each duration has a different default for the burn rate threshold.

Unfortunately, users aren't always aware that burn rate thresholds should be tightly coupled with the SLO duration. They may also be unaware of the industry standards for burn rate thresholds for each lookback window for the various SLO durations.

This issue is furthered by our automatic burn rate rule creation. While automatic out of the box rule creation improves the user experience overall, users are now unaware of the connection between their SLO params and the burn rate rule configuration (they may even be unaware a burn rate rule exists).

This means that users are unaware that they should update their burn rate rule configuration when changing the SLO duration.

To account for this, we could consider the following.

  1. Auto updating burn rate rules. Since we auto create the rules, we could consider also auto updating them. We should check the rule associated with the SLO to see if it has been customized or remains as the default configuration. If it remains in the default configuration, auto update the burn rate rule to utilize the correct burn rate thresholds.

  2. A callout whenever the SLO duration is changed. We could add a callout underneath the SLO duration dropdown when updated to inform the user to update their SLO burn rate thresholds. We could then auto open the burn rate threshold flyout to allow for editing their rule. This is likely the only option for rules that have been customized by the user.

elasticmachine commented 5 months ago

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

jasonrhodes commented 5 months ago

@grabowskit can you weigh in on this, priority-wise?