Elasticsearch Template Add cluster_name tag to elasticsearch_cluster_health, elasticsearch_cluster_health_indices measurement

va2dim commented 1 year ago

Use Case

I need cluster_name tag in this measurement (elasticsearch_cluster_health, elasticsearch_cluster_health_indices) for filtering dashboard for 2 or more projects with Elasticsearch under this dashboard.

Expected behavior

in this measurement elasticsearch_cluster_health, elasticsearch_cluster_health_indices add tag "cluster_name" and with change of cluster_name in elasticsearch config - passsing correct cluster_name

Actual behavior

in this measurement (elasticsearch_cluster_health, elasticsearch_cluster_health_indices) now we have only tag "name", that always bring value "elasticsearch" (what is it? it is no cluster_name, no host). And when I have 2 project with Elasticsearch I cannot take metrics under this measurements. And for all other measurements we have tag "cluster_name".

Additional info

No response

powersj commented 1 year ago

Hi,

The elasticsearch_cluster_health metric's tag name is set to the cluster name:

https://github.com/influxdata/telegraf/blob/master/plugins/inputs/elasticsearch/elasticsearch.go#L405-L410

The elasticsearch_cluster_health_indices metric's tag name is also set the cluster name:

https://github.com/influxdata/telegraf/blob/master/plugins/inputs/elasticsearch/elasticsearch.go#L424-L429

va2dim commented 1 year ago

Hi,

The elasticsearch_cluster_health metric's tag name is set to the cluster name:

https://github.com/influxdata/telegraf/blob/master/plugins/inputs/elasticsearch/elasticsearch.go#L405-L410

The elasticsearch_cluster_health_indices metric's tag name is also set the cluster name:

https://github.com/influxdata/telegraf/blob/master/plugins/inputs/elasticsearch/elasticsearch.go#L424-L429

That what I think also. And expected behavior if I change cluster.name in config (elasticsearch.yaml) this tag give me this changed value. But this not happen (always same values).

In all others measurements we have tag cluster_name, and it work as expected (if I changed also variable elasticsearch_cluster_name to schema.tagValues( bucket: v.bucket, tag: "cluster_name", predicate: (r) => r._measurement == "elasticsearch_clusterstats_nodes", start: v.timeRangeStart ) ) , but in elasticsearch_cluster_health_indices, elasticsearch_cluster_health - not.

I just need same tag cluster_name for all measurements, for the purpose use this dashboard not only for few different elasticsearch servers.

Also when I get Cluster Heath metrics from elasticsearch (/_cluster/health?level=indices and /_cluster/health) I have correct cluster_name.

powersj commented 1 year ago

We won't be changing the tag key name from name to cluster_name. Changing tags can lead to unexpected behavior and additional cardinality. You can make the change yourself with starlark or rename the tag processor.

[[processors.rename]]
  namepass = ["elasticsearch_cluster_health", "elasticsearch_cluster_health_indices"]
  [[processors.rename.replace]]
    tag = "name"
    dest = "cluster_name"

va2dim commented 1 year ago

We won't be changing the tag key name from name to cluster_name. Changing tags can lead to unexpected behavior and additional cardinality. You can make the change yourself with starlark or rename the tag processor.
[[processors.rename]]
  namepass = ["elasticsearch_cluster_health", "elasticsearch_cluster_health_indices"]
  [[processors.rename.replace]]
    tag = "name"
    dest = "cluster_name"

I try this, but problem not in naming, but under this tag "name" I didnt see values that expected - cluster.name from elasticsearch.yaml ( always pass value "elasticsearch"), also the same with tag "host" and only under this measurements ("elasticsearch_cluster_health", "elasticsearch_cluster_health_indices") - always pass value "elastic" (and not real host name like tag "host" under other measurements).

What is a source of data for this tags (look like passing always same data)?

powersj commented 1 year ago

What is a source of data for this tags (look like passing always same data)?

I've already posted links to the code where this data is collected. In all cases we take the JSON we receive and parse out the cluster_name value.

Cluster health itself is gathered from /_cluster/health

influxdata / telegraf