Additional Labels not exported by KSM

[bhanugarg23]

What happened:

I am not able to add additional pod and deployment labels into the metrics exported by KSM. I have setup KSM via helm inside AWS EKS. Below is the change I made in the values file to add allow the additional label for pods

metricLabelsAllowlist:
- pods=[deplEnv]

Here deplEnv is the label key. When I apply this change in the values file, I am seeing this extra args being passed in the deployment. When I port forward the KSM service I am not able to see this label in any of the metrics on the /metrics page.

What you expected to happen:

Label with the key deplEnv visible in the pod and deployment metrics

How to reproduce it (as minimally and precisely as possible):

Apply the kube-state-metrics helm chart in AWS EKS(version 1.27) with the change mentioned above in the values file and then port forward the service

Anything else we need to know?:

Environment:

• kube-state-metrics version: 2.10.1
• Kubernetes version (use kubectl version): 1.27
• Cloud provider or hardware configuration: AWS EKS
• Other info:

[grzesuav]

Whic metric does not have this label exactly ?

[bhanugarg23]

None of the metrics have this label. When I port forward and open the /metrics page on local, I am not find this label anywhere on the page

[bhanugarg23]

I got the issue. KSM was relabelling the metric name deplEnv to label_depl_env. I am only able too see this label on metric kube_pod_labels and not on any other metric. Is there a way I can see this label across all metrics for the pod ?

[dgrisonnet]

I got the issue. KSM was relabelling the metric name deplEnv to label_depl_env. I am only able too see this label on metric kube_pod_labels and not on any other metric. Is there a way I can see this label across all metrics for the pod ?

That is expected, the labels are only added to the _labels metrics.

/assign /kind support

[dgrisonnet]

/triage accepted

[thefirstofthe300]

I too would like to see this particular feature added to kube-state-metrics. I'm trying to figure out how to get a metric time series which shows the versions of pods currently deployed. I figured I could do this by enforcing a version label being set on the pods, but without the ability to add my own labels to a metric, it becomes very difficult to do.

[ronaknnathani]

Having this feature would be super helpful! We enforce an app label on each pod where the label value is equal to the user defined app names. Having these labels show up on other metrics would significantly improve usability.

[ronaknnathani]

Sent a PR to solve this for pods - https://github.com/kubernetes/kube-state-metrics/pull/2428. If the reviewers agree on the approach, can update it for all the resources.

[dgrisonnet]

This feature request came up quite a few times already and was always rejected because we want to stick to a 1:1 mapping between Kubernetes objects and the metrics. Any processing on the data is something that should be done outside of kube-state-metrics. For instance you can already achieve what you are trying to do by aggregating the _labels metric with your metric.

You also have to consider that if we were to add labels to all the metrics that would serious impacts on storage considering the sheer amount of data that would represent to essentially just store the same information over and over. Because of that, aggregation is preferred.