sustainable-computing-io / kepler

Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe performance counters and other system stats, use ML models to estimate workload energy consumption based on these stats, and exports them as Prometheus metrics
https://sustainable-computing.io
Apache License 2.0
1.19k stars 184 forks source link

With EXCLUDE_SWAPPER_PROCESS enabled, kepler reports VM power more than actually consumed by VM #1836

Open sthaha opened 3 weeks ago

sthaha commented 3 weeks ago

Setup

Deploy latest kepler with EXCLUDE_SWAPPER_PROCESS set to true on a baremetal

kepler-1  | I1105 04:34:34.940824  939885 config.go:267] EXCLUDE_SWAPPER_PROCESS: true

Test

image

Note that the

Blue (which is fraction of the total power) is reported way higher than the increase in kepler-node-package watts In the above example, the increase in watts reported by kepler_node_package is as below image

It raise from 18.0 to 29.2 =~ 12 W

While kepler_vm_package watts is as follows

image

It raises from 0.9 to a max of 26.9 W

Additionally, ` abs( sum(rate(kepler_process_package_joules_total{job="metal"}[20s])) - sum(rate(kepler_node_package_joules_total{job="metal"}[20s])) )


 shows that the difference is sometime as high as 6W 

![image](https://github.com/user-attachments/assets/2431d058-5ec7-4177-88a0-35dcc05bd39b)