Open Dosant opened 3 years ago
Pinging @elastic/kibana-app-services (Team:AppServices)
Please see this google doc for a list of all the possible scenarios and views that the UI needs to account for and confirm these are all correct
@Dosant Since you're only focused on aggregations, I tested this by looking at the bsearch
responses that Lens is getting. It seems like we aren't getting any partial aggregation results on 7.11 where I tested. I'm not sure if this is a bug or expected, but if we can't get partial data, all we can do is show a loading indicator: https://github.com/elastic/kibana/issues/94495
@wylieconlon partial results were exposed on SearchSource
only in 7.12, but they were available in lower level search since 7.10 I think.
If you're just looking at the responses, try focusing on really large data sets and long running queries. I'm working with a 30 million document data set (30% of it is in frozen indices) and I get partial results returned.
Let me know if you need more info. :pray:
@lizozom I think I do need more information, since all I saw was that the hits
count was increasing. What data is returned during the partial aggregations? Which buckets? Maybe you could post a sample that you generated?
Sharing our current thinking on this for feedback:
Below are two iterations. We have some slight variations on these that I can share, but these seemed to be the most receptive. With both of these we are also doing a slight fade during chart transitions to help avoid flashing/jumping. I'm faking it with this prototype, so please let me know if there is something else we need to be aware of for this.
I will also add these to this doc so we can gather feedback a bit easier.
@mdefazio I have some comments on what you're showing that I couldn't find a place in the doc to comment on:
Since date histograms are a very common chart type, it would be nice to show this more realistically- for example, by having only the data on the left or right of the chart visible.
Thanks for the comments @wylieconlon , I'll try and update to show more realistic time-series examples
I took a look at the API responses and noticed that the search is 99% done (by number of shards), so it's only waiting for the last 1% before displaying the results. In our case it's querying multiple clusters via CCS, so one of those clusters might be slow. It would be great if it could display these partial results, but with a visual indicator that these results are not complete. (the last 1% is probably not going to change the view completely)
{"took":262313,"timed_out":false,"terminated_early":false,"num_reduce_phases":98,"_shards":{"total":39736,"successful":39681,"skipped":33506,"failed":0},...
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)
Agreed that some functionality to show in progress results would be great. For me the use case is that I occasionally do large queries over long time windows (though not with CCS). As henrikno states, it would be nice if each or some batched number of shards responses could be rendered as they are available before they all complete.
Part of https://github.com/elastic/kibana/issues/84051
We need a common way to represent a partial result state. Probably, it would overlap with the search session UI. It could be visually connected with it.
We will use this state on a dashboard on each panel that supports partial results:
related: https://github.com/elastic/kibana/issues/76521 (we should get rid of or improve this desaturation) and https://github.com/elastic/kibana/issues/55408