[Custom threshold] Show no data setting in the rule's flyout

maryam-saeidi commented 3 months ago

Summary

Previously, we've decided to simplify Custom threshold's rule creation flyout and only allow adjusting this setting via API. Recently, we found out that this can cause an issue in cases where the user does not set a no data action but has an action for the recovered action group. In this case, the user will get a notification for a recovered no data alert but hasn't received any alert notification before that. (slack)

In order to help user fix this issue in UI, we would like to bring back the no data setting.

Questions

@maciejforcone, Can you please share your input about where would be the best place to show this setting in the rule's flyout?

The current location of this setting in the metric threshold rule does not look good because we have 2 advanced options on the flyout. Maybe a better option would be to keep this setting close to missing group and add a documentation link for clarification.

Current location in Metric threshold rule:

Based on this discussion on slack, we would like to keep the no data setting "unchecked" by default in UI.

elasticmachine commented 3 months ago

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

maryam-saeidi commented 3 months ago

@heespi @doakalexi proposed to show a warning in case no data is configured, but we don't have a no data action configured for it, @jasonrhodes @vinaychandrasekhar what are your thoughts about it?

The only challenge that I see is explaining the scenario that sending alert and no data to different actions will lead to confusion since the recovered action is shared between the two groups. Maybe we shouldn't allow having separate action groups for no data. 🤔

Example

Alert -> channel-1 Recovered -> channel-1 No data -> channel-2

In this case, when no data alert is recovered, users will see a recovered notification in channel-1 without receiving alert notification, and channel-2 will not get recovered notification at all. (Related ticket)

jasonrhodes commented 3 months ago

@heespi @doakalexi proposed to show a warning in case no data is configured, but we don't have a no data action configured for it, @jasonrhodes @vinaychandrasekhar what are your thoughts about it?

I think this is a good idea. I think this is confusing enough that we need a little UI help. (We could consider sending a warning message back in the API too, but I don't think we should block the rule creation without this.)

The current location of this setting in the metric threshold rule does not look good because we have 2 advanced options on the flyout. Maybe a better option would be to keep this setting close to missing group and add a documentation link for clarification.

I think we should not have these two different Advanced Settings accordions, so moving these two together (inside a single Advanced Settings or not, I don't mind which one really) sounds like the right choice to me.

jasonrhodes commented 3 months ago

See note about "advanced settings" plan here: https://github.com/elastic/kibana/issues/183921#issuecomment-2145285270

maryam-saeidi commented 1 month ago

After a discussion with @maciejforcone, to improve this situation, we decided to implement this setting as mentioned here. The related improvement was implemented in https://github.com/elastic/kibana/pull/188300.

We still have the following question to answer:

In case we enable a no data setting and do not have any no data action configured, do we want to show a warning? (related to this comment) If yes, how would we want to show it in UI, and do we want to also cover it in API?

elastic / kibana