search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.98k stars 24.75k forks source link

Ability to monitor usage of runtime fields #90348

Open ckauf opened 2 years ago

ckauf commented 2 years ago

Description

From an Elastic Stack administrator point of view it is desirable to monitor the usage of runtime fields. Based on the metrics an admin can make the recommendation to move the calculation of runtime fields into the ingest pipeline instead of calculating the values on-the-fly/during a search.

Today the field usage stats API does not provide this insight and as far as I know these metrics do not exist anywhere else in the stack.

elasticsearchmachine commented 2 years ago

Pinging @elastic/es-search (Team:Search)

javanna commented 2 years ago

We have the field usage stats API that tells you how fields are used, but that is only for indexed fields and not for runtime fields. It would be nice to have a similar functionality for runtime fields to assist making decisions around schema evolution. Note that for runtime fields that need to be made indexed, it is not necessary to use an ingest pipeline. The runtime field can be moved from the runtime section to the properties section of the mappings, see https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime-indexed.html , in that case the script is going to be executed at ingest time and the result is going to be indexed.

elasticsearchmachine commented 3 months ago

Pinging @elastic/es-search-foundations (Team:Search Foundations)