grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.27k stars 172 forks source link

How to add the container label for cadvisr #830

Closed bingkunyangvungle closed 3 months ago

bingkunyangvungle commented 4 months ago

URL

https://github.com/grafana/alloy/blob/9f6bc3a1ab553fd25c439dab1b148cb0736e59ba/docs/sources/reference/components/prometheus.exporter.cadvisor.md

Feedback

I'm currently using prometheus.exporter.cadvisor "cadvisor" to get the related metrics. The metric can show up, but when I was trying to add the labels using the following configuration:

allowlisted_container_labels = ["io.kubernetes.container.name", "io.kubernetes.pod.name", "io.kubernetes.pod.namespace"]

The label doesn't show up in the metrics. I've already set this: store_container_labels = false because in the doc, it mentioned:

For allowlisted_container_labels to take effect, store_container_labels must be set to false.

Could anyone help to look into this and provide some guideline? Thank you.

ptodev commented 4 months ago

Could this be because the labels contain dots? Prometheus labels are quite strict with what characters are allowed in them. Could you try a label such as io_kubernetes_container_name?

bingkunyangvungle commented 3 months ago

Thank you @ptodev , I just tried with your suggestion io_kubernetes_container_name and it still returns error:

Error: /etc/alloy/config.alloy:82:37: component "io_kubernetes_container_name" does not exist or is out of scope
81 |     store_container_labels = false
82 |     allowlisted_container_labels = [io_kubernetes_container_name]
   |                                     ^
83 | }

Not sure what this one was. And it's not in the doc as we.

ptodev commented 3 months ago

You're missing the quotation marks :)

allowlisted_container_labels = ["io_kubernetes_container_name"]
bingkunyangvungle commented 3 months ago

Oh my, this is the cause. Really appreciate for your help! @ptodev

bingkunyangvungle commented 3 months ago

The issue resolves. But the metrics doesn't appear neither. Still need more investigation.

ptodev commented 3 months ago

Hi @bingkunyangvungle, out of curiosity - how did you resolve the issue?