Twice similar jobs with eck operator

[domcar]

Hi everyone,

i recently installed a ES Cluster using the eck operator and then install the prometheus exporter to get metrics. Everything works without problems, only I get the same result twice: For example, if I query prometheus with: es_os_mem_used_bytes I get :

es_os_mem_used_bytes{cluster="elastic-jaeger", container="elasticsearch", endpoint="https", instance="10.43.11.105:9200", job="elastic-jaeger-es-http", namespace="eck-cluster", node="elastic-jaeger-es-masters-0", nodeid="RCaOVA6vRCOrAwWN7wJaBg", pod="elastic-jaeger-es-masters-0", service="elastic-jaeger-es-http"}

es_os_mem_used_bytes{cluster="elastic-jaeger", container="elasticsearch", endpoint="https", instance="10.43.11.105:9200", job="elastic-jaeger-es-masters", namespace="eck-cluster", node="elastic-jaeger-es-masters-0", nodeid="RCaOVA6vRCOrAwWN7wJaBg", pod="elastic-jaeger-es-masters-0", service="elastic-jaeger-es-masters"

where the only difference is the job name, this makes Grafana a bit messy. The names seems to be the same as the services exposed by the cluster.

Anyone know what I did wrong or point me to the right direction?

[mindw]

@domcar The above suggests you have two services for the pod and both are scraped for metrics.

[domcar]

Yes, I have two services, but they were deployed automatically with the elastic search operator, not sure if I can simply remove one

[mindw]

just one of the scraping configs needs to be removed. How does the prometheus scraping of the service configured?

[domcar]

I have a serviceMonitor with a labelsSelector, but unfortunately the two services coming from the eck-operator have the same labels :( So I can change them manually and it's fine, but I can't find how I can modify them using the operator

[domcar]

Maybe this is more an issue on the operator side and not here, unless you used it too and know how to handle this