elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.57k stars 24.62k forks source link

Expensive search stats updates with many groups #85586

Open jtibshirani opened 2 years ago

jtibshirani commented 2 years ago

The search section of the index stats API allows search stats to be grouped by custom tags. To tag what group a search belongs to, the search API supports a stats parameter.

In our implementation ShardSearchStats, the grouped stats are stored in an immutable map. Every time a new group is added, we duplicate the whole map. So if there are a large number of individual groups, the updates can be really expensive and can even affect search latency.

We should look into making these updates less costly, as well as maybe bounding the number of groups that we attempt to track.

elasticmachine commented 2 years ago

Pinging @elastic/es-data-management (Team:Data Management)