[Custom threshold] Simplify rule creation flyout for logs use cases

[maryam-saeidi]

🍒 Summary

In the initial discussion with @gbamparop and @ruflin to integrate the Custom threshold rule in the Logs Explorer view, they pointed out that the current UI for the Custom threshold rule creation is complicated and not suitable for new users. The same feedback was mentioned by the infra team, and for a similar reason, we have an Inventory threshold rule that simplifies what can be done in the Metric threshold rule.

The goal of this is ticket to start a discussion with the logs team and come up with a design that is simpler and more intuitive for new users, but also giving the user the option to see the more advanced UI if they want to.

Here is what this rule looks like currently:	Condition section	Grouping section

[elasticmachine]

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[maryam-saeidi]

@gbamparop @ruflin I had a chat with @maciejforcone and he prepared a mockup for the simplification in this Figma:

Before	After

The challenge here is to understand what are exactly the logs use cases that our users use, is only having a KQL bar with filters enough to show initially and the rest will be shown when a user selects the advanced mode? What are your thoughts about this topic?

I will check to see if there is any data about this topic. If you know someone who might know about our rule usage, please let me know :)

[ruflin]

I like the simplification. As you describe, it would be nice if users could know what is selected (at least for now). I don't know how to put it best into the design but something like Data: FROM logs-* as text somewhere on the top. I'm sure design has some good ideas (@isaclfreire ). The advanced button is nicely hidden. I was thinking if we should potentially move it under the two conditions? The thought is a user is looking at the conditions and thinking: I want to do more and has directly there the advanced button.

[gbamparop]

@thomheymann @isaclfreire any thoughts about this? Let's look into what is there to move it forwards.

[isaclfreire]

I like where this is going, because feels much less complex than before. Agree with @ruflin on showing explicitly the data source selected (and could even me interesting for it to be changeable if possible?). I would also argue that it would be nice to change rule type eventually if needed, but maybe this isn't in scope now. Also, it's important that depending on how the user gets to this flyout (I'm thinking from Logs Explorer, for instance), we also make sure to keep the context of the query the user already created, if it was the case.

[gbamparop]

Also, it's important that depending on how the user gets to this flyout (I'm thinking from Logs Explorer, for instance), we also make sure to keep the context of the query the user already created, if it was the case.

If you mean the KQL query in Logs Explorer, it's being preserved already when opening the flyout.

[thomheymann]

This is great, thank you for looking into this!

Instead of creating another bespoke data source / filtering UI would it be possible to use the data selector from Logs Explorer? It’s familiar to users and provides better integrations support. The only fields it does not support is defining rule conditions but I see that as a separate step anyways.

In addition to the bar chart and threshold marker could we show the outcome of the rule to the user? For example where and when an alert would be generated?

How does ES|QL fit into this redesign? From my understanding it's the direction of travel so would be good to cater for that.

I’m not a fan of hiding settings behind “advanced options” toggles since it provides no hints to the user about what options can be found there. Instead I would let users select these options in context (e.g. the "WHEN COUNT" label could be a dropdown menu allowing users to select whether they want to define their own custom equation). Alternatively we could also show more specific toggles (e.g. “Customize rule equation” and "Add filters" instead of a single "More options")

[jasonrhodes]

I’m not a fan of hiding settings behind “advanced options” toggles since it provides no hints to the user about what options can be found there.

I agree with this for places where we want the user to be able to bounce back and forth between some kind of "simple" mode and an "advanced" mode for an individual UI element. I do think it's possible (and okay) that we might end up with something that also allows for a more complete "SIMPLE MODE" overall, vs "ADVANCED MODE", where we hope that most users don't need to use the advanced one and it's only there as an escape hatch. In that context, I think it would be okay to hide those things from most users, I think. Let's see how things develop.

Alternatively we could also show more specific toggles (e.g. “Customize rule equation” and "Add filters" instead of a single "More options")

I like this idea, where it makes sense.

[weltenwort]

Instead of creating another bespoke data source / filtering UI would it be possible to use the data selector from Logs Explorer? It’s familiar to users and provides better integrations support.

@maryam-saeidi has already proven this to work in general in a POC. It currently requires some undesirable workarounds, but we could clean that up on the data selector side if we choose to go ahead with this.

[maciejforcone]

Hi, sharing updates after our session with @maryam-saeidi

[maryam-saeidi]

@maciejforcone While discussing the new design with @jasonrhodes, he pointed out an idea about possibly having Add aggregation and set equation as one item together, not sure how it might look like but it would be interesting to discuss. I also created a ticket to discuss how we can simplify the equation section for future infra use cases.

[jasonrhodes]

Yes, I love this direction but I find it really hard to understand what the difference is between "Add another condition" vs "Add aggregation" vs "set custom equation" -- and since the aggregations are only needed when you are using custom equation, it would be nice (imo) to just treat those as a single editing experience somehow. I also don't know exactly how this would look in practice, but I'd love to explore options for making that cleaner in the "More options" section...