search

robscott / kube-capacity

A simple CLI that provides an overview of the resource requests, limits, and utilization in a Kubernetes cluster
Apache License 2.0
2.09k stars 114 forks source link

Total utilization vastly exceeds sum of pods #100

Open joshfrench opened 1 year ago

joshfrench commented 1 year ago

I'm seeing this pattern reliably across a number of clusters. The total utilization doesn't seem to make any sense given the individual pod metrics. Requests and limits always add up to their respective totals which makes me wonder if the total cpu & mem are based on their own metric and not just summed. Do you have any insight into what those are actually measuring? Looking at the obvious suspects it doesn't seem to line up with the working set, etc.

Thanks!

CPU UTIL     MEMORY UTIL

275m (13%)   4729Mi (60%)
52m (2%)     102Mi (1%)
4m (0%)      58Mi (0%)
50m (2%)     219Mi (2%)
1m (0%)      18Mi (0%)
0m (0%)      173Mi (2%)
2m (0%)      23Mi (0%)
1m (0%)      70Mi (0%)
1m (0%)      7Mi (0%)
6m (0%)      45Mi (0%)
2m (0%)      30Mi (0%)
almson commented 7 months ago

I agree this is confusing. It doesn't even match what htop is showing. However, kubernetes separates the concepts of allocatable resources (excluding those used by the OS) and total resources. Since we're only really interested in allocatable resources, it would be more consistent if the sum of mem util would be the sum of the pods' mem util, excluding the OS. There could be a separate column or warning for the OS and its actual vs reserved resource usage.