kubernetes / kube-state-metrics

Add-on agent to generate and expose cluster-level metrics.
https://kubernetes.io/docs/concepts/cluster-administration/kube-state-metrics/
Apache License 2.0
5.2k stars 1.92k forks source link

kube_pod_spec_volumes_persistentvolumeclaims only report the first PVC when there are more than one PVC mounted to the same Pod. #2397

Closed yummydsky closed 1 month ago

yummydsky commented 1 month ago

What happened: There are two volumes mounted on the same pod, but only the first volume can be found in kube-state-metrics.

kubectl get pvc -n influxdb
NAME                                         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
influxdb-data.pvc                 Bound    pvc-544ce610-d4f8-4ade-ade7-ea43046e4f1a   100Gi      RWO            local-path     351d
influxdb-log.pvc                  Bound    pvc-f1d0ced2-f717-4c7e-abdf-c69c978b7164   2Gi        RWO            local-path     351d

But in Prometheus

sum(kube_pod_spec_volumes_persistentvolumeclaims_info) by (pod, namespace, persistentvolumeclaim, volume)
{namespace="influx", persistentvolumeclaim="influxdb-data.pvc", pod="influxdb-0", volume="influxdb-data-storage"}

--


What you expected to happen:

The metric kube_pod_spec_volumes_persistentvolumeclaims_info can report all the volumes mounted information not just first one.

How to reproduce it (as minimally and precisely as possible):

Step 1. Create a pod with two persistent volume
Step 2. Check the kube-state-metrics kube_pod_spec_volumes_persistentvolumeclaims_info could report these two volumes or not.

Anything else we need to know?:

Environment:

k8s-ci-robot commented 1 month ago

This issue is currently awaiting triage.

If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
ricardoapl commented 1 month ago

Hello @yummydsky,

I can't reproduce your issue with either v2.10.1 or v2.12.0 (latest):

kube-state-metrics /metrics

kube_pod_spec_volumes_persistentvolumeclaims_info{namespace="default",pod="task-pv-pod",uid="317729a4-7dfa-4f53-8569-a3245e6a6e75",volume="task-pv-storage",persistentvolumeclaim="task-pv-claim"} 1
kube_pod_spec_volumes_persistentvolumeclaims_info{namespace="default",pod="task-pv-pod",uid="317729a4-7dfa-4f53-8569-a3245e6a6e75",volume="task-pv-storage-2",persistentvolumeclaim="task-pv-claim-2"} 1

PromQL

{namespace="default", persistentvolumeclaim="task-pv-claim", pod="task-pv-pod", volume="task-pv-storage"}
{namespace="default", persistentvolumeclaim="task-pv-claim-2", pod="task-pv-pod", volume="task-pv-storage-2"}

Kubernetes

Client Version: v1.30.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.0

Any chance you could try a more recent version of Kubernetes? v1.21 reached End of Life 2022-06-28 [1]

[1] https://kubernetes.io/releases/patch-releases/#support-period

yummydsky commented 1 month ago

@ricardoapl Thanks for the quick reply. Yes, you're right. This is not a bug. That's my fault. It's my pod template issue, I just requested the volume but did not mount it to a specific folder, so the metric works as expected.