When using limited privilege environment (roles) empty metrics endpoint

[SennaSemakula]

What happened: I ran kube-state-metrics using Roles and RoleBindings following: https://github.com/kubernetes/kube-state-metrics?tab=readme-ov-file#limited-privileges-environment but I did not pass in --namespaces as it should default to all namespaces including the current one. When I navigate to localhost:8080/metrics the endpoint is blank.

What you expected to happen: When using limited privileges (roles and not clusterroles) it should have metrics on the current namespace. Note this works if you do pass in --namespaces but it should work regardless as the default value monitors all namespaces based on: https://github.com/kubernetes/kube-state-metrics/blob/main/pkg/options/options.go#L146

How to reproduce it (as minimally and precisely as possible):

1. Follow steps but make sure you do not specify --namespaces as it should default to all namespaces by default
2. Navigate to localhost:8080/metrics and you should see no metrics

Anything else we need to know?: I'm working on deploying kube-state-metrics in a tenanted environment that is using: https://github.com/kubernetes-sigs/hierarchical-namespaces.

Environment:

• kube-state-metrics version: 2.12.0
• Kubernetes version (use kubectl version): v1.27.12-gke.1115000
• Cloud provider or hardware configuration: GKE
• Other info:

[k8s-ci-robot]

This issue is currently awaiting triage.

If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

[thunderbird86]

Have the same issue, but I didn't setup any limited privileges environment, it works fine for some time and now it has the same empty responce

[Serializator]

The problem originates from the use of v1.NamespaceAll, where KSM assumes it can list and watch resources at the cluster scope. This is a wrong assumption in a limited privilege environment.

W0623 13:07:43.643600  104637 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.2/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kube-state-metrics" cannot list resource "pods" in API group "" at the cluster scope

https://github.com/kubernetes/kube-state-metrics/blob/main/internal/store/builder.go#L514-L524

The thing is though, without the permission to list namespaces, KSM can't determine what namespaces we have access to either. The minimal requirement in a limited privilege environment for something like this to work is a cluster role (binding) to allow the listing of namespaces across the cluster.