search

canonical / grafana-agent-operator

https://charmhub.io/grafana-agent
Apache License 2.0
4 stars 10 forks source link

possible regression in node exporter metric label for juju unit should include principal charm unit and not the grafana agent unit #152

Closed nishant-dash closed 2 months ago

nishant-dash commented 2 months ago

Bug Description

using rev 134 of grafana-agent I can see metrics from node exporter where the juju_unit label is the grafana agent unit and not the principal

node_cpu_seconds_total{agent_hostname="juju-63a125-microk8s-0", cpu="0", instance="juju-63a125-microk8s-0", job="juju_microk8s_2ed63743-be7d-455c-8428-63d93763a125_grafana-agent-microk8s_node-exporter", juju_application="grafana-agent-microk8s", juju_model="microk8s", juju_model_uuid="2ed63743-be7d-455c-8428-63d93763a125", juju_unit="grafana-agent-microk8s/0", mode="idle"}

however an older rev 28 does this correctly

node_cpu_seconds_total{agent_hostname="microk8s-3", cpu="0", instance="microk8s_37ded2f1-3840-4590-88a5-eefeda03885a_microk8s_microk8s/2", job="juju_microk8s_37ded2f1-3840-4590-88a5-eefeda03885a_grafana-agent-microk8s_node-exporter", juju_application="microk8s", juju_model="microk8s", juju_model_uuid="37ded2f1-3840-4590-88a5-eefeda03885a", juju_unit="microk8s/2", mode="idle"}

I have not thoroughly tested this, so I can not say for sure if its a regression or a race condition.

To Reproduce

N/A

Environment

N/A

Relevant log output

N/A

Additional context

No response

lucabello commented 2 months ago

I believe this is the intended behavior. A VM can be shared by multiple principles: in that case, Grafana Agent wouldn't know which principal to use in the labels.

I think that at the time we weren't supporting multiple principles, and thus we made a different choice for the labels, but it's hard to tell!

Closing this, but please feel free to reopen if you have anything to add!

nishant-dash commented 2 months ago

Hey @lucabello from a usability perspective this is quite difficult because if I need to look at metrics for, lets say, nova-compute/10, then I need to do the dance to find the corresponding grafana agent unit id and look at that instead Is it not possible to inject the principal charm topology when there is only 1 principal and inject grafana topology otherwise?