search

kubernetes / kube-state-metrics

Add-on agent to generate and expose cluster-level metrics.
https://kubernetes.io/docs/concepts/cluster-administration/kube-state-metrics/
Apache License 2.0
5.4k stars 2.01k forks source link

To obtain the "Allocated resources:" from the output of kubectl describe node <node-name>, what query should be used? #2522

Open githubeto opened 2 weeks ago

githubeto commented 2 weeks ago

I believe that the most accurate value for determining whether a Pod can be scheduled is the Requests section under "Allocated resources:" obtained from kubectl describe node .

image

This is because if these values are close to 100% (for example, 99%), attempting to schedule a Pod will result in a resource shortage, making it impossible to schedule.

So, how can we obtain this value using a Prometheus query in Grafana?

The reason I am asking this question is that while the query kube_node_status_allocatable{resource="memory"} can be used to obtain the total capacity of the node, the sum of the memory requests of currently scheduled Pods obtained with sum by (node) (kube_pod_container_resource_requests{resource="memory"}) clearly exceeds the node's capacity. Each of my nodes has 16Gi of memory, but this query returns a total that exceeds 16Gi.

18.2Gi ...? (Worker node is 16Gi memory spec.)

k8s-ci-robot commented 2 weeks ago

This issue is currently awaiting triage.

If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.