search

prometheus-community / elasticsearch_exporter

Elasticsearch stats exporter for Prometheus
Apache License 2.0
1.91k stars 792 forks source link

Feature request - Watermark and read_only indices #310

Open frenkye opened 4 years ago

frenkye commented 4 years ago

Hi,

great exporter, but Iam missing variable for watermark and percent usage of node.

1) Node: curl localhost:9200/_cat/allocation?h=node,disk.percent 2) Watermark: curl localhost:9200/_cluster/settings if not present in persistent or transient settings, then 85% is defaut.

More about watermark is here https://www.elastic.co/guide/en/elasticsearch/reference/7.5/disk-allocator.html

Because if ES hit the flood_stage (introduce in version 6, before high was the limit), then all indices are switch to read only and have to be manually unlock AFAIK, they planned to do auto unlocking in disk space is freed, but don't know if that is implemented in new versions.

With this would be also helpfull if there was variable for tracking number of read only indices, this can be found under /_all/_settings if set index.blocks.read_only_allow_delete: true

For example like was issue here https://github.com/elastic/kibana/issues/13685

stefnestor commented 1 month ago

Because if ES hit the flood_stage (introduce in version 6, before high was the limit), then all indices are switch to read only and have to be manually unlock AFAIK, they planned to do auto unlocking in disk space is freed, but don't know if that is implemented in new versions.

Confirming from Elastic's flood-stage troubleshooting guide, the read_only_allow_delete block automatically removes after node leaves flood-stage+high watermark thresholds. Implemented v7.4.0 via elasticsearch#42559.