search

siderolabs / talos

Talos Linux is a modern Linux distribution built for Kubernetes.
https://www.talos.dev
Mozilla Public License 2.0
6.6k stars 527 forks source link

Talos VM breaks energy saving on Intel / Proxmox #8848

Closed Skaronator closed 4 months ago

Skaronator commented 4 months ago

Bug Report

Description

I'm currently transforming my homelab to a low-power setup. I noticed that running Talos Control Plane within a VM skyrocket the power consumption of the whole server. I did some A/B testing using just an Ubuntu VM, and it doesn't increase in power consumption, as you can see here:

image

This is a freshly installed system. Using latest Proxmox, latest Talos running just a single VM.

Ubuntu and Talos VM are identically configured:

CPU Usage is pretty similar. Ubuntu at 1% and Talos at 2-3%.

Server Idle: ~ 15W Ubuntu 24.04 Server VM: ~19W Talos ISO Boot: ~50W Talos bootstrapped as Control Plane (out of 3): ~50W

Logs

Environment

Skaronator commented 4 months ago

I did another test using Debian 12 which has an older kernel (6.1). I did install gnome by mistake, so CPU usage is a bit higher at 4% (and higher than Talos VM) but power consumption is similar to Ubuntu 24.04 at ~20W.

DmitriyMV commented 4 months ago

@Skaronator is there any additional info or issue or we can close this?

Skaronator commented 4 months ago

Hi @DmitriyMV,

thanks for the response. Here is some additional information that I can provide:

Here is the talos configuration. It currently has only 3 Control Planes and VIP enabled: https://github.com/Skaronator/homelab-talos

The bare-metal machine was freshly installed using Proxmox VE 8.2 ISO Installer.

The VM was started using the Talos v1.7.4/metal-amd64.iso image with the configuration I already provided above:

After the first startup, I joined the VM to my existing (but also fresh, empty) Talos cluster.

talosctl apply-config --insecure -n 192.168.1.47 --file out/final-cp-3.yaml

Hardware:

Logs are probably not that helpful because there are no errors. (Would be weird on a fresh vanilla setup)

image

image (I did a couple of reboots, that why it spiked a few times)

smira commented 4 months ago

Are you comparing Ubuntu VM with Kubernetes installed or just a VM with nothing running?

tzabbi commented 4 months ago

@Skaronator is there any additional info or issue or we can close this?

@DmitriyMV I think this is a really crucial issue related to the current situation of cost saving and climate protection. My employer uses GreenIT as an important evaluation point, which technologies we can use or not.

rothgar commented 4 months ago

Can you try upgrading to an image that has the Intel firmware and ucode extensions from factory.talos.dev and let us know if the power consumption goes down

Skaronator commented 4 months ago

@rothgar thanks for suggesting the additional intel firmware extensions, totally forgot them.

Using these options:

customization:
    systemExtensions:
        officialExtensions:
            - siderolabs/i915-ucode
            - siderolabs/intel-ice-firmware
            - siderolabs/intel-ucode

(as well as setting this option on the host cpupower frequency-set --governor powersave) fixes it!

rothgar commented 4 months ago

Glad it worked for you. Can you share what you're using to measure the power usage? I really like that dashboard and might look into something similar.

I'm also curious if Talos is less than the Ubuntu VM in your environment. I have been wanting to test this on bare metal to compare if the defaults on different OSs affect power consumption in any meaningful way.

Skaronator commented 4 months ago

I use Shelly Plus Plug S to measure directly from the wall. The dashboard is Home Assistant. It's not as good as Prometheus with Grafana, but does plenty of other things well :)