Chart loading status

[markov00]

Should we need add a loading status/indicator on the chart?

Depending on the data, the chart can take a few seconds to appear the first time with a big number of elements, for small/standard time it's a matter of milliseconds (we still have to fix https://github.com/elastic/elastic-charts/issues/109 to reduce the time for the first render)

There is also other factor that influence the loading time of a chart, independent of the chart implementation like any http request, data processing or everything else needed to get the right data and configuration to startup a chart.

I'm thinking of a simple flag on the Chart component or as a simple empty component that can be used in the following cases:

• we want to display a loading icon when we are waiting for the data to arrive (use case: dashboard)
• we want to keep displaying that loading icon when computing the geometries of the chart, before rendering

Two ideas:

// using a prop like ready to specify that the chart can be intialized 
// in an empty state and start showing a loader
<Chart ready={false}>
</Chart>

// or
<Chart>
   <NotReady />
</Chart>

// Then when you have your data, just remove that flag. We will maintain internally 
// a ready flag to false until we are not ready to render, enabling the ready flag
// the loading will be removed from the chart element
<Chart>
   <LineSeries ... />
</Chart>

Chart ready prop	geometries computed	loading indicator
❌	❌	⏳
✅	❌	⏳
✅	✅	📊

What do you think? it's something that the chart needs to provide or we leave this to the UI using the chart?

If we implement that, we can easily align all the loading indicators on every code that use our library

ping @emmacunningham @emmacunningham

[emmacunningham]

@markov00 It seems like a good idea to have the chart provide these indicators; as you mentioned, this allows us to visually align all of the loading indicators for users of the library. To your point, there are two states to track here: the data loading and the chart computations. Maybe there can be an option for users to specify a custom loading screen if they prefer to use their own instead of the chart internal version (similar to how users would be able to customize tooltip elements).

I think the first option of having the user set the ready prop is a simple interface to be able to signal this state.

[cauemarcondes]

On 7.9.0 we are going to release the new Observability Dashboard page (https://github.com/elastic/kibana/issues/68176), the page consists of multiple charts, where I'd really appreciate to have a loading indicator to be shown while I'm fetching the data. For the meantime, I wrapped the Chart in a container which shows the EuiLoadingChart until the data is fetched. But I'd rather use a default visualization.

[monfera]

One challenge with a chart loading indicator is that if it's prominent, then an entire dashboardful of charts turn into a spinner circus :-) The Canvas team ran into this. So even if it exists - whether it's the chart itself or the containing unit - it's better to make it subtle like in http://www.nytimes.com/newsgraphics/2014/02/14/fashion-week-editors-picks/index.html (scroll downwards and see momentary progress indicators) and/or separate the notion of blanking out or blurring some charts vs. showing a progess indicator, maybe for the entire dashboard.

Other challenges exist too, eg.

• an instance of elastic-chart may not be the best unit for a progress indicator
• progress is made even before elastic-chart is invoked, ie. besides its control; would the container unit eg. dashboard roll its own, perhaps different progress indicator?

Given that ES is good at quickly giving a partial response and less quickly giving a full response, there's also the eventual option of incremental rendering of the data.

[sorenlouv]

+1 for loading indicators on charts. I agree with @monfera that it's better to keep it subtle, and the NY Times example is great because it is very similar to how EUI Table loading indicator looks like

NY Times

EUI Table

Similar to the EUI Table loading indicator it should be possible to indicate that new data is being loaded, while still displaying the old data. This is particularly important when auto-refresh is enabled and we want to avoid the content "glitching" everytime a new fetch is initiated.

Thus, I think there are two loading states:

• The initial loading state when no data is available to display. In this case we might want to have an empty state in addition to the loading indicator
• Subsequent loading states (initial data has already been loaded but the user clicked "refresh" or changed the time picker). In this case a loading indicator is enough.

[formgeist]

After my conversation with @markov00 today, I went ahead and quickly mocked up a few examples of adding a small progress indicator to the chart itself and a small "forecast" indication in the chart, when the new data has yet to be rendered and the existing data fills the appropriate space. This might work as an intermediate transform from the original dataset to the new.

I've used the existing EuiProgress component for the indicator but because of the styling (background vs. foreground) and the minimum size still being a bit tall, we might consider creating another variation or creating one specifically for the chart container.

The next steps for me would be to reach out to other teams that might have a need for something like this and see if this fills their requirements, e.g. the Dashboards or Security teams.

Example: Full page with multiple charts


➡ View Figma prototypes

PS: Hope the potato quality of the GIFs didn't ruin the visuals too much 🥔

[sorenlouv]

Awesome @formgeist! This will be a great improvement to the APM charts at least.

[ryankeairns]

+1 to many of the points shared here.

With regards to Dashboard, I'm thinking a determinate progress bar (if possible; like the one in Søren's example) might be a nice way to manage the visual noise (aka the 'circus' that Robert mentions) were we to go this with the proposed solution above. Please loop me into any discussions with the Presentation Team. Thanks for moving this along!

[sorenlouv]

This would still be incredibly useful. I have recorded a small example from APM where latency is displayed as a line chart, and the expected bounds are displayed as an area chart.

When changing the time range the latency chart shows up quickly, and Expected bounds take a little longer. Without a loading indicator the user will not know that the visualisation is incomplete and might move on.

https://user-images.githubusercontent.com/209966/171942381-be2fbe3c-31e2-4134-acda-fde6dee299da.mp4

Another problem I've found myself is that I've trained myself to wait for 10-15 seconds because I expect that the visualisation is incomplete - often only to find out that everything has indeed finished loading and there's nothing missing.

[sorenlouv]

Perhaps the best way to move forward is to create a simple container around our visualisations that provides a loading indicator like @formgeist suggested in https://github.com/elastic/elastic-charts/issues/202#issuecomment-738802525.

I don't think this is technically difficult. The downside is a fragmented experience since different solutions might solve this in different ways.

[monfera]

Another tricky bit is that as it is, elastic-charts is still a "big bang" type of charting library (React-inspired, data->chart model), which you initialize once you have all the required data on hand. IOW charts has no idea about the period spent in waiting. @markov00 may have other comments, this is my initial view:

Short term: charts is still big bang; components that use charts can wrap each chart instance (or collectively, say, for an entire dashboard, or a subgroup in a dashboard, as they wish) in an almost identically sized container except for the subdued NYT/EUI-like loader bar. Benefits: can be done now; the chart embedder has the info; can be adapted for the look and feel of the containing page, and the sensible granularity (eg. if a page has a 4x4 grid of charts, individual loaders may look overly noisy, overanimated and ultimately unhelpful)

Longer term: we have a theme of chart deconstruction. This would allow a sequence such as:

• the chart initially receives key aggregate data, such as pixel sizes, x domain and maximum y value; maybe, categories
• the chart renders everything except the actual data
• the embedder keeps sending completion % info, allowing the chart to show a progress bar, OR better: the embedder keeps sending chunks of data (from pagination, or the initial results come from pseudorandom samples—cc @flash1293) OR maybe the previous data context is close enough so that a y domain is kept around while loading (eg. the filters just shifted time, and we have data in a chart or embedder cache)

Much of the longer term thing can already be approximated well today: just keep restating the chart with whatever info is already available. For example, if the embedder can quickly get the x and y domains, then an empty chart can already be rendered, helping the user (only the bar/line data will be new for the user; the user minimizes chart reading latency by familiarizing and contextualizing the chart scaffolding)

[monfera]

PS. a minor note is that we keep enhancing charts with annotations, an example for which is the recent WebGL flame alpha (try it here) which eg. indicates focal area with rendered line sections. So I agree that rendering a line is not hard, and that we're sympathetic with the need for rendered metadata like that, as it enriches the user experience.

However we also have the types of questions pondered by y'all: should it be obligately sourced from EUI, and is that component ready for this, now, and for the future; should charts depend on more and more EUI components for displaying things

[flash1293]

For example, if the embedder can quickly get the x and y domains, then an empty chart can already be rendered, helping the user

It really depends whether this information can be obtained quickly. Sometimes these things are specified as part of the chart configuration or by dashboard level configuration (global time range), but sometimes they depend on the data itself. For the first case we can leverage this information without too many changes, for the second case which would require additional requests sent out together with the "actual" data requests this means quite a bit of extra complexity.

[ryankeairns]

Reminder: iirc, we intentionally removed the per-panel loaders for the sake of dashboard. If we add them back, perhaps we can optionally disable/not show them at a dashboard level. Instead, we settle on some other global solution there. cc:/ @mdefazio

[monfera]

I agree @flash1293 there is a tradeoff between "time to first data-driven render" and query+code simplicity. And there's no guarantee that the more aggregated domain min/max queries, or unique category queries finish any sooner than the detailed data query (or queries)

[markov00]

Perhaps the best way to move forward is to create a simple container around our visualisations that provides a loading indicator like @formgeist suggested in #202 (comment).

I agree @sqren, this looks like the quickest way to solve the problem, this doesn't bring an extraneous component into elastic-charts, doesn't force a solution by default, and doesn't bring the need of extending this everytime a team, designer, engineer needs a different loading spinner due to a different situation.

I don't think this is technically difficult. The downside is a fragmented experience since different solutions might solve this in different ways.

As with everything else around in Kibana, no one is forcing the use of a specific style (for better or for worse). The use of EUI is a guideline, and the use of elastic-charts too, you can always bring your own style, change colors, sizes etc, but we always try to push for good defaults.

If we implement this within elastic charts, we need to provide a way to enable/disable this (there are many cases a user doesn't want to display an horizontal loading spinner). When the disable option is available, is clear that someone will use it and apply a different loader, even if one is available.

Also, we should consider that there are multiple use cases that require probably different strategies:

• initial loading of charts
• subsequent data loading
• partial data loading
• loading one chart vs loading a page with a few charts or a dashboard with multiple charts
• chart size (does sparklines in a table count the same way it count for a full-width time chart?)

That said, we can think of a short term plan for the current @sqren use case) with the following steps:

• ask design (aka @gvnmagni @formgeist @ryankeairns etc) to understand the best style of partial loadings
• check with EUI if we have already a loader with the defined characteristics (horizontal infinite bar for example) and if they are ok to have the design/implementation/maintenance ownership (The Kibana login page already have such loader indicator but I don't see it in EUI)
• When the component is ready, we can create a sharable/reusable wrapper in Kibana
• we then should write on bot EUI and our chart documentation what are the best practices for loading indicators in charts, providing the link to the reusable component
• and finally hope for a global adoption across teams with the same need

WDYT?