elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.61k stars 8.22k forks source link

[Maps] Support most recent document by entity to be numeric field #46882

Closed Alex3k closed 5 years ago

Alex3k commented 5 years ago

Describe the feature: The "Show most recent document by entity" feature within the maps app currently only takes a datetime field. It would be really valuable to support a numeric field also. For example the largest or smallest (configurable) value in Field A.

Describe a specific use case for the feature: To get maximum value from drilling (oil and gas) data, I am encouraging companies to ingest all the raw numeric (time & depth series) data into Elasticsearch. This way they can leverage the power of our aggregations and visualisations to easily get value without much effort. A single wellbore can have 250,000 documents, sometimes more. Many organizations have 100k well bores. This data is indexed by depth (float value).

As we only plot the first N documents on a map, this often leads to a map only having one wellbore present. The "Show most recent document by entity" is almost perfect. If it supported the above request, we could only show only one wellbore for all of the documents without much effort at all. Currently, I am suggesting to use transforms (previously known as data frames) to summarise all the wells as they enter Elasticsearch.

elasticmachine commented 5 years ago

Pinging @elastic/kibana-gis

bcamper commented 5 years ago

I think this would be a great addition. But I also wanted to point out that this specific case (and others like it) are likely to also run into the 10k aggregations limit (since the "per entity" is implemented as a terms aggregation), e.g. if the "group by" (100k wells) is over 10k. I've run into this with other datasets.

@nreese do you know if we have an issue logged for addressing that limit? If not I'll add one.

nickpeihl commented 5 years ago

I believe this issue addresses the 10k aggregation limit. https://github.com/elastic/kibana/issues/27606