elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.76k stars 8.16k forks source link

[ML] Change point detection in Lens #176714

Open darnautov opened 7 months ago

darnautov commented 7 months ago

Currently change point detection functionality (based on the change_point agg) in Kibana is limited to the AIOps labs and embeddable either attached to a dashboard or configured by other plugins programmatically.

image

High-level overview

We perform the analysis on the time series data, where the user has to configure a metric and a function, and optionally a split field. When a split field is defined, we utilize the composite agg for partitioning by that field. Then for each partition (or a single metric, if a split field is not set up), run the change_point agg to retrieve the list of actual change points.

Then using the change point date we plot the metrics with the Lens embeddable and render change points as an annotation layer.

TODO

We can make it available for a wider audience by incorporating it into Lens - the go-to visualization tool.

Presumably, it can be implemented as a dedicated layer type. If it's feasible, it should not require additional attribute types for Lens.

On top of that, we can introduce a UI action to quickly attach the change point detection layer to existing Lens panels.

elasticmachine commented 7 months ago

Pinging @elastic/ml-ui (:ml)

darnautov commented 7 months ago

cc @stratoula

stratoula commented 7 months ago

@darnautov I am intrigued! What do you mean with new layer type?

It seems as something we could fit on the breakdown functions?

image

Can you give me some use cases and examples of visualizations which could benefit from this aggregation?