prometheus-community / elasticsearch_exporter

Elasticsearch stats exporter for Prometheus
Apache License 2.0
1.94k stars 791 forks source link

Snapshot metrics - not finding if the last backup was successful #961

Open GeiserX opened 2 weeks ago

GeiserX commented 2 weeks ago

I enabled the --collector.snapshots to see which metrics I could get from the snapshots I'm doing in my ES cluster. However these are the only metrics I found related to it then:

elasticsearch_scrape_duration_seconds{collector="snapshots"} 0.006456413
elasticsearch_scrape_success{collector="snapshots"} 1

And then many times this structure, for each node:

elasticsearch_thread_pool_active_count{cluster="deduplicator-es",es_client_node="true",es_data_node="false",es_ingest_node="false",es_master_node="true",host="...",name="deduplicator-es-es-master-2",type="searchable_snapshots_cache_fetch_async"} 0
elasticsearch_thread_pool_active_count{cluster="deduplicator-es",es_client_node="true",es_data_node="false",es_ingest_node="false",es_master_node="true",host="...",name="deduplicator-es-es-master-2",type="searchable_snapshots_cache_prewarming"} 0
elasticsearch_thread_pool_active_count{cluster="deduplicator-es",es_client_node="true",es_data_node="false",es_ingest_node="false",es_master_node="true",host="...",name="deduplicator-es-es-master-2",type="snapshot"} 0
elasticsearch_thread_pool_active_count{cluster="deduplicator-es",es_client_node="true",es_data_node="false",es_ingest_node="false",es_master_node="true",host="...",name="deduplicator-es-es-master-2",type="snapshot_meta"} 0

I would simply like to check if the last backup failed or not, but there's nothing I can check related to that it seems. I have set one daily backup.

I don't know if this is a feature request, a bug, or if I'm doing something wrong on my side, honestly.

Thank you

Skunnyk commented 1 day ago

Once enabled, you should have tons of metrics related to snapshots: https://github.com/prometheus-community/elasticsearch_exporter?tab=readme-ov-file#metrics like elasticsearch_snapshot_stats_xxxxxx