The value of jobLabel in a ServiceMonitor defines the label of the scraped service whose value should be used as job_name by Prometheus. However, it looks like in the chart it was being used as if it was meant to be the actual value one wanted to use for the job. Because of
this, scraped metrics would end up with, say, fluentd-elasticsearch-metrics (the name of the scraped Service) instead of fluentd-elasticsearch.
This is particularly tricky since it would interact badly with the provided PrometheusRules. For example, FluentdNodeDown, that would normally be triggered when there are no fluentd pods running, gets defined as up{job="fluentd-elasticsearch"} == 0, but because there are no
samples generated for that job, it remains forever green!
We now use app.kubernetes.io/instance as jobLabel and make it overridable in the Values.yaml.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]
What this PR does / why we need it:
The value of
jobLabelin aServiceMonitordefines the label of the scraped service whose value should be used asjob_nameby Prometheus. However, it looks like in the chart it was being used as if it was meant to be the actual value one wanted to use for the job. Because of this, scraped metrics would end up with, say,fluentd-elasticsearch-metrics(the name of the scrapedService) instead offluentd-elasticsearch.This is particularly tricky since it would interact badly with the provided
PrometheusRules. For example,FluentdNodeDown, that would normally be triggered when there are no fluentd pods running, gets defined asup{job="fluentd-elasticsearch"} == 0, but because there are no samples generated for that job, it remains forever green!We now use
app.kubernetes.io/instanceasjobLabeland make it overridable in theValues.yaml.Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]
[fluentd-elasticsearch])