search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.71k stars 8.12k forks source link

[Lens] Sliding window support #131073

Open flash1293 opened 2 years ago

flash1293 commented 2 years ago

To better deal with time series data, it should be possible to specify a "window size" for a metric dimension which corresponds to how much data is aggregated for each time bucket. This decouples the resolution of the time axis (the step size) from the range from which data is pulled in.

This configuration can happen transparently for the user - if a "window size" is configured on the metric, Lens automatically switches to the sliding date window, keeping the existing auto interval logic for the step interval while using the configured window size for the window interval.

Screenshot 2022-04-27 at 16 57 34

A longer window is smoothing out the value and can be used to handle gaps more elegantly (e.g. even if the resolution of the chart falls to second, the counter rate calculation is still taking into account from a minute ago so there will always be values.

Different window sizes for multiple metrics in the same layer require a way to filter metric aggs to a given range relative to the bucket bounds.

elasticmachine commented 2 years ago

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

jtkarb commented 1 year ago

This would be extremely helpful. For example if you today want to display moving_average over a period of one hour you need to set the window parameter according to you current time bucket duration. If my chart has 10-minute buckets I need to set the window=6, but if the user then changes the overall data period displayed in the dashboard, then the chart may not display 10-minute buckets anymore but maybe 30-minute buckets and then window size is wrong for calculating moving_average for a one-hour period. Also, this suggestion would enable a moving_sum calculation, which is currently missing in Lens.

On a side note, this would also address the issue mentioned in #151000, where is can be seen that the quick function for moving_average does not observe a certain time window. If for example you have 10-minute buckets and set window=6 the actual period used for the calcultion may be longer than one hour as it just uses the last 6 non-zero buckets for the calculation (as opposed to the formula functions, which includes the non-zero buckets and respects a time period = window * bucket duration).