[kube-proemtheus-stack] kube-state-metrics resource verticalpodautoscalers

[youwalther65]

Describe the bug a clear and concise description of what the bug is.

I have installed the following kube-state-metrics version

$ helm list -n prometheus
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                                         APP VERSION
kube-prometheus-stack   prometheus      1               2023-07-14 13:37:52.1355345 +0200 CEST  deployed        kube-prometheus-stack-48.1.1                  v0.66.0

kube.state-metrics "CrashLoopBackOff" with log message:

$ k get all -n prometheus
NAME                                                            READY   STATUS             RESTARTS      AGE
pod/alertmanager-kube-prometheus-stack-alertmanager-0           2/2     Running            0             51s
pod/kube-prometheus-stack-grafana-785fbc6c9c-57c8x              3/3     Running            0             53s
pod/kube-prometheus-stack-kube-state-metrics-7c7f47fcbf-pgqbp   0/1     CrashLoopBackOff   2 (27s ago)   53s
...

$ k logs -n prometheus kube-prometheus-stack-kube-state-metrics-7c7f47fcbf-pgqbp
I0714 11:39:16.999457       1 wrapper.go:98] "Starting kube-state-metrics"
W0714 11:39:16.999659       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0714 11:39:16.999787       1 server.go:198] "Used resources" resources=[replicationcontrollers resourcequotas endpoints limitranges persistentvolumes poddisruptionbudgets pods leases mutatingwebhookconfigurations storageclasses volumeattachments certificatesigningrequests deployments statefulsets verticalpodautoscalers configmaps cronjobs horizontalpodautoscalers networkpolicies nodes namespaces secrets validatingwebhookconfigurations daemonsets services ingresses jobs persistentvolumeclaims replicasets]
E0714 11:39:16.999811       1 wrapper.go:40] "Failed to run kube-state-metrics" err="failed to set up resources: resource verticalpodautoscalers does not exist. Available resources: clusterrolebindings,endpointslices,networkpolicies,statefulsets,validatingwebhookconfigurations,certificatesigningrequests,daemonsets,mutatingwebhookconfigurations,persistentvolumeclaims,services,storageclasses,pods,roles,rolebindings,horizontalpodautoscalers,ingresses,leases,namespaces,nodes,replicationcontrollers,ingressclasses,limitranges,persistentvolumes,poddisruptionbudgets,clusterroles,configmaps,deployments,endpoints,replicasets,resourcequotas,serviceaccounts,volumeattachments,cronjobs,jobs,secrets"

It complains with err="failed to set up resources: resource verticalpodautoscalers does not exist but the CRD is installed by VPA Helm:

$ k api-resources | grep vertical
verticalpodautoscalercheckpoints          vpacheckpoint            autoscaling.k8s.io/v1                    true         VerticalPodAutoscalerCheckpoint
verticalpodautoscalers                    vpa                      autoscaling.k8s.io/v1                    true         VerticalPodAutoscaler

VPA Helm chart is:

$ helm list -n vpa
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                         APP VERSION
vertical-pod-autoscaler vpa             1               2023-07-14 08:33:50.90701707 +0000 UTC  deployed        vertical-pod-autoscaler-7.2.0 0.14.0

What's your helm version?

v3.10.1

What's your kubectl version?

v1.27

Which chart?

kube-prometheus-stack

What's the chart version?

48.1.1/v0.66.0

What happened?

KSM pod in CrashLoopBackOff

What you expected to happen?

KSM pod to get into running state

How to reproduce it?

Run helm install with the below non-standard values.yaml

Enter the changed values of values.yaml?

I enable VPA in Helm values.yaml with:

$ cat values.yaml
..
kube-state-metrics:
  namespaceOverride: ""
  rbac:
    create: true
# add VPA to KSM
  collectors:
    - certificatesigningrequests
    - configmaps
...
    - verticalpodautoscalers
    - volumeattachments
    - ...

and it has worked for quire a while

Enter the command that you execute and failing/misfunctioning.

$ helm install kube-prometheus-stack --namespace prometheus prometheus-community/kube-prometheus-stack -f values.yaml

Anything else we need to know?

N/A

[Bregor]

@youwalther65 looks like you are using kube-state-metrics with version >=2.9.0. Please look at the deprecation notice: https://github.com/kubernetes/kube-state-metrics/blob/v2.8.2/docs/verticalpodautoscaler-metrics.md#deprecation-notice

[dotdc]

@youwalther65 Can you confirm the issue is resolved, so we can close this issue?