search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.9k stars 24.73k forks source link

ESQL: Use filters by default when calling fields caps #115053

Open costin opened 5 hours ago

costin commented 5 hours ago

Description

Currently, calls to field caps API don't use a filter in order to get access to all data and not avoid false negatives - however this is expensive since the entire mapping has to be read. To improve performance we can use an optimistic approach by invoking the API with an index filter and, in case not all fields are found issue a second call without a filter. Overall this will be more expensive than one call however we're estimating that:

  1. the first call will succeed more often than not
  2. the first call will be cheaper
  3. the delta between the first and second call, gives us insight around fields are excluded by the filter which can be at planning time.
elasticsearchmachine commented 5 hours ago

Pinging @elastic/es-analytical-engine (Team:Analytics)