Kibana should be able to execute queries without field list details

mattkime commented 8 months ago

Currently, kibana expects to have the full field list before making a request. Getting the full field list in large data configurations with a lot of shards and clusters can be very slow - multiple seconds or longer. The field list is then used to construct the query in a way that ES expects and the results are useful for Kibana.

It should be possible for Kibana to execute requests WITHOUT inspecting the field list and specific fields. This will require changes to the ES DSL. We will need to mind cross cluster search compatibility before relying on such changes. For a variety of reasons this will be a long term project but it should be worthwhile for increased performance and data efficiency reasons.

TODO

[ ] look through /packages/kbn-es-query/src for fields references. These are likely cases where conditionals could be removed.

Incomplete list of tasks

[ ] https://github.com/elastic/elasticsearch/issues/103705

elasticmachine commented 8 months ago

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

kertal commented 5 months ago

related to https://github.com/elastic/kibana/issues/175081 should we consolidate?

kertal commented 6 days ago

I think this is no longer up to date, resolved and can be closed, feel free to reopen if you think differently

elastic / kibana

Kibana should be able to execute queries without field list details #173966