[Lens] Custom Breakdown Presentation Order

[MichaelMarcialis]

As part of the One Discover contextual logs effort, we would like to present a histogram of documents broken down by the log.level field. To make this information easier to interpret, it would be ideal if we could have greater control over the breakdown presentation order, allowing us to order the legend and breakdown stacking by log.level severity (see example in mockup below). As I understand it, this is currently not possible on either the Lens and Elastic Charts side.

Related issue: https://github.com/elastic/kibana/issues/86184

CCing @stratoula, @teresaalvarezsoler, @nickofthyme.

[elasticmachine]

Pinging @elastic/kibana-visualizations (Team:Visualizations)

[teresaalvarezsoler]

@stratoula is this chart only for ES|QL or also for the dataview mode?

[stratoula]

Both! right @davismcphee ?

[teresaalvarezsoler]

Scope for this issue: we need a bit more investigation to figure out how to solve this problem so we are repurposing this as a spike and time-boxing it to < 1 week

[davismcphee]

@stratoula correct! This is needed for both data view and ES|QL mode.

[nickofthyme]

This functionality will be available in kibana when https://github.com/elastic/kibana/pull/195447 is merged.

[markov00]

https://github.com/elastic/kibana/pull/195447 is merged but this can be achieved today only through Lens override functions, that are not persisted into Lens nor into a Dashboard. I'd like to propose adding this feature in Lens by reusing the sorting order of the color mapping for rendering purposes too. The sorting order of the data is inherently connected to the breakdown. This should be part of the breakdown configuration, where you can specify the data sorting (not the ranking): usually you can specify it with asc/desc/numerical/alphabetical but you can also specify it via in a list/index of terms/values to follow. For now, we can probably also consider using the color mapping for that functionality, because it aligns with that concept. This can be applied on a request basis via a switch in the color mapping like the following example:

In the long run, probably these operations should be described in the "data processing" part of Lens and not connected to the data styling.

[MichaelMarcialis]

I'd like to propose adding this feature in Lens by reusing the sorting order of the color mapping for rendering purposes too.

@markov00: I agree that we should provide configuration controls for users to apply a custom order to their visualization breakdowns. However, I do have a few concerns about relegating that control to the color mapping interface:

1. I fear it will have very low discoverability there. What if a user doesn't wish to change their breakdown colors (which I believe is a majority of users)? Would they know to find such sorting controls there?
2. Using a breakdown dimension isn't the only way for users to stack bar or area charts. For example, what if a user has multiple stacked bar layers with no breakdowns? Would we need to accommodate for adjusting the sort order in such a scenario? If so, I don't think the color mapping interface would be viable, as it is attributed to a single layer.
3. The current color mapping interface has no drag-and-drop support for the reordering of terms. We can of course add drag-and-drop to it, but I fear that it would make an already complex UI even more complex.

Also, this related issue from Wylie seemed to indicate that there were use cases beyond just breakdowns.

Personally, my initial instinct is to add any sort of sorting controls at the global visualization level (i.e. toolbar menus), if we need to account for that multi-layer example I describe above. If we don't have to account for the multi-layer example, then at the root-level of the appropriate layer dimension's configuration flyout (i.e. not within the deeper color mapping configuration) would be my preference. Thoughts?

[markov00]

@MichaelMarcialis let's discuss this offline and understand what/where we can go