[Lens] Synchronize cursor position for X-axis across all Lens visualizations in a dashboard

wylieconlon commented 4 years ago

Description of feature

Users expect to use dashboards to compare their data interactively, such as by hovering over one chart and seeing the related charts update. This type of synchronized cursor has been supported by many parts of Kibana, but Lens has not supported this yet. It's currently supported in TSVB, Stack Monitoring,

One example is in the Stack Monitoring application, where each of these charts has the same X axis. In this chart, there are no tooltips, and the legend is used to show the value at the current time:

Elastic-Charts has a storybook example which uses tooltips instead of updating the legend, which might work better for Lens:

How would this get exposed in Lens?

My preference is to make this behavior automatic for as many Lens charts as possible, so that users get this behavior by default. I would imagine that we can synchronize not just time series data, but any data on the X-axis. For example, if the user has built a dashboard for ecommerce data, we could synchronize all the product categories on the X axis. The default behavior would be to show the tooltips, not to update the values in the legend.

If we do want to allow some user choices, I imagine that there are the following choices users could make:

Show the currently hovered value in the legend (off by default)
Disable synchronized tooltips (on by default)

Implementation options

TSVB's implementation of synchronized cursors uses a mutable object called eventBus. Lens could use the same approach, which would create a separate cursor state for Lens and TSVB visualizations.

Alternatively, Lens could implement a set of UI actions that are triggered on hover, and these actions could be responded to by other Lens visualizations.

Finally, if we use the system of UI actions, we could potentially synchronize both Lens and TSVB visualizations.

elasticmachine commented 4 years ago

Pinging @elastic/kibana-app (Team:KibanaApp)

flash1293 commented 4 years ago

That's pretty cool and I can imagine it being super helpful for complex dashboards. Does this work across partition charts / cartesian charts as well?

@elastic/kibana-app-arch Do you think UI actions are the right primitive to implement such a feature, or do we need a separate one?

wylieconlon commented 4 years ago

I took a look at the implementation of external pointer events in the charts library, and it seems like pie charts might be able to use it, but I didn't confirm. Maybe @markov00 can comment?

markov00 commented 4 years ago

@wylieconlon @flash1293 the current mechanism works only on XY axis charts and it's not implemented in Pie charts.

a pie chart doesn't have a cursor over an axis but we can just highlight categories
gauge/goal charts instead can probably have a cursor that works across its range (not yet implemented)
heatmaps can have selectable ranges on X and Y
treemaps, as for pie, can highlight categories or group of categories

In most cases, they can't share their cursors across different charts, if not through filtering (see https://square.github.io/crossfilter/) or highlighting linked series (series with the same metric and attributes like: the pie is split by country, and we have a line chart that shows data by countries when hovering over the US slice on the pie we can highlight the US line on the line chart).

This feature is similar to the palette/color synch between charts as we are again on the concept of sharing attributes between charts: color, positions, etc are all sharable attributes that need careful consideration when shared: you don't want to sync the cursor/tooltip of two histograms just because both x-axes are a continuous numeric scale but you probably want to connect only charts with the same metric units and attributes like the in the following example: a dashboard with 3 histograms: # COVID infections by ages, # death by ages, hospitalization. In this situation, all 3 histograms have a continuous x-axis that goes from 0 to 100 (ideally) but we should sync only the first two because you are interested in cross-comparing infections and deaths by ages. The number of days the infection took to be "resolved" is a parameter that we don't want to sync with ages. We probably want to highlight as done by crossfilter only a portion of the last histogram that covers the age range/value currently selected: histograms

We can't simply rely on series names (and I think we can't rely on series names also for colors too) because of stylistic reasons or readability reasons I can use different names for one and the other chart/metric/series. We also can't assume that every chart with a time scale needs to sync its tooltip with all the other time scale charts.

My idea is that we should provide, at a dashboard level, the ability to link those attributes manually between charts, the user should be in the power of connecting these attributes and synch them in the way he found more useful for the use case. I'd like to stress the fact that this needs to be done at the dashboard level because it can be possible that I'm using the save viz on multiple dashboards, but I'm using different configuration per dashboard (the same happens for colors) Lens should provide the ability to accept events from a dashboard or any other embedding environment, but the environment should provide the right links

wylieconlon commented 4 years ago

@markov00 Thanks for the detailed response. I'm not sure I understand this part of the comment:

We also can't assume that every chart with a time scale needs to sync its tooltip with all the other time scale charts.

Syncing all the time scales was one of the behaviors I was assuming because it's such a commonly used dimension in all of our charts. What's wrong with showing tooltips on unrelated charts that all have time scales?

timroes commented 4 years ago

I think as part of this issue, we should not sync only within Lens, but make sure we connect those two systems. So with whatever system we're coming up with (e.g leveraging uiActions) we should imho also leverage that in TSVB afterwards, and have it in a generic way that would allow other implementations also to connect to it.

markov00 commented 4 years ago

Syncing all the time scales was one of the behaviors I was assuming because it's such a commonly used dimension in all of our charts. What's wrong with showing tooltips on unrelated charts that all have time scales?

@wylieconlon I'm saying that because I think it should be up to the user to decide if a time-based chart needs to be in sync with other time-based charts or not. On TSVB it's forced by default, but it can be frustrating if on a dashboard you have multiple dashboard and you are exploring them and all the tooltip and cursors moves all around all together. If instead of providing a default on not changeable option as first step we can think of a better solution that already provides ways to disable/enable sharing cursors along charts on a dashboard it can give much more value to the user. The dashboard modal can provide a "good" default: sync every time-based chart together with a way to select just a few charts to sync together and no sync option.

wylieconlon commented 4 years ago

@markov00 I like your suggestion of making this controllable at the dashboard level, and keeping this behavior on by default but with an option to disable. I think we can break this into a few steps of implementation:

Decide on the syncing mechanism and any limitations
Make changes to TSVB so that it can sync with Lens and other future charts
Allow the dashboard to disable the TSVB time-syncing behavior
Implement in Lens following the TSVB pattern

markov00 commented 4 years ago

I'm fine with the plan One side note: my idea on the dashboard settings was slightly different and and includes 3 ways to handle syncing:

sync tooltip on every time-based chart
no sync at all
sync only selected time-based charts

this, in the long rung, should pave the way to other sync mechanism like: sync colors, sync sorting by, etc

GrahamHannington commented 3 years ago

Really looking forward to this. When I'm developing a dashboard that contains more than one time-based chart, I'm deliberately using TSVB instead of Lens for all of them, because Lens doesn't show that vertical red line.

Any estimate on when this feature will be released?

flash1293 commented 3 years ago

Thank you for reaching out @GrahamHannington . There are no short-term plans to implement this, however use engagement (like your comment) are definitely an important factor to give features a priority bump. For Lens, you can get a rough idea of what the team is working on by looking at the project board: https://github.com/elastic/kibana/projects/32

cc @ghudgins

GrahamHannington commented 3 years ago

Thanks for the encouragement, and for the link to the board 👍.

It's interesting (to me) that the Kibana UI specifically promotes Lens as the first choice for visualizations—it's the tile at the top left, with the text "Recommended for most users"—whereas the Kibana docs topic "Create panels with editors" presents Lens first, but as if among equals, with no such "this is your go-to viz type" text.

I suggest that most users adding multiple time-based charts to a dashboard would opt for TSVB as their first choice if they knew about that vertical red line.

Suggestion for the short-term: in the Kibana docs topic "Supported features by panel type", perhaps under the subheading "Other features", add a row that describes this vertical red line and related behavior, with a check mark under TSVB (and any other columns that apply), but not under Lens.

flash1293 commented 3 years ago

Implementation notes: For TSVB this is handled by a subject instance which is kept in global plugin state:

https://github.com/elastic/kibana/blob/a900110a344b2be4bd2d3fc0b89379691a84a830/src/plugins/vis_type_timeseries/public/application/visualizations/views/timeseries/index.js#L76

https://github.com/elastic/kibana/blob/a900110a344b2be4bd2d3fc0b89379691a84a830/src/plugins/vis_type_timeseries/public/application/visualizations/views/timeseries/index.js#L51

https://github.com/elastic/kibana/blob/4584a8b570402aa07832cf3e5b520e5d2cfa7166/src/plugins/vis_type_timeseries/public/application/visualizations/lib/active_cursor.ts#L12

We should change this like that:

Move the activeCursor subject into the charts plugin and expose it via contract
Make TSVB consume it from there
Do the same integration TSVB uses (onPointerUpdate, chartRef.current.dispatchExternalPointerEvent) in Lens as well

tbd: Do we want to expose a switch for this on dashboards? Not sure whether necessary - cc @ghudgins

ghudgins commented 3 years ago

@flash1293 I don't think it's necessary given I can't find anyone asking to turn this behavior off in TSVB today (could be wrong about that though...just did a quick search)

~that being said if it's easy to add we're not at a place where we have too many dashboard settings so it might be nice to have~

1 july 2021: we decided to make it on by default

flash1293 commented 3 years ago

Just discussed offline - As soon as timelion switched over to elastic-charts, we can use the same mechanism there and have Lens, TSVB and Timelion cursors sync.

ghudgins commented 3 years ago

discussions from sync:

we should align the line/area stroke format across timelion / TSVB / lens (gray, solid stroke until @markov00 tells us otherwise)
time can be "global" with regard to highlighting across different index patterns on a dashboard
non-time fields are data / index pattern specific with regard to highlighting

ghudgins commented 3 years ago

@alexwizp on advice from @markov00 we should go with the EUI chart theme default (dashed line). remove the custom style and that should do it. the design team can evaluate and change it centrally if they want to make updates

matschaffer commented 3 years ago

Is there a way to get the popup to follow as well? If not maybe I'll open an issue for it. I can see the cursor tracking across graphs, but it's not as useful to have to eyeball the Y axis value.

I'd be okay if this was an option that need to be enabled, not on by default. But when you need it, you need it :)

Internet_speeds_-_Elastic

markov00 commented 3 years ago

@flash1293 I think we should talk and open a discussion with the @elastic/kibana-presentation to configure this at the dashboard level, rather than at a chart level. The user should be able to select when and where to display the shared cursor and tooltip in the dashboard

flash1293 commented 3 years ago

Agreed, @matschaffer could you open a separate issue? Thanks for binging this up, this would indeed be a very nice feature.

matschaffer commented 3 years ago

Sure thing! I opened up https://github.com/elastic/kibana/issues/113719 - feel free to adjust as needed

elastic / kibana