search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
86 stars 4.92k forks source link

[Metricbeat] Elasticsearch module should only gather node_stats it needs #40183

Open henrikno opened 3 months ago

henrikno commented 3 months ago

Describe the enhancement: Metricbeat elasticsearch module is currently fetching all node_stats metrics, even though a lot of them are not used. https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html It's recommended to only ask for what you use. Some can be expensive, e.g. allocations.

Describe a specific use case for the enhancement or feature: In 8.14 _node/stats started requiring querying the master node to get the allocations stats. https://github.com/elastic/elasticsearch/pull/105894 Since we run metricbeat elasticsearch module on each ES node, this adds a lot of requests to the master node. But we're not collecting this stat so we can just omit gathering it.

consulthys commented 2 months ago

@henrikno @jguay thanks for your contributions! Would it make sense to combine both of your PRs together since they both include the same metrics list? https://github.com/elastic/beats/pull/40184 seems to be a little bit more restrictive than https://github.com/elastic/beats/pull/40664 since it also includes some indices metrics.