siderolabs / talos

Talos Linux is a modern Linux distribution built for Kubernetes.
https://www.talos.dev
Mozilla Public License 2.0
6.65k stars 531 forks source link

Talos does not upgrade after applying a machineconfig with a different installer #9438

Closed spastorclovr closed 1 week ago

spastorclovr commented 1 week ago

Bug Report

When reading :point_right: https://www.talos.dev/v1.8/talos-guides/configuration/system-extensions/#example-booting-from-an-iso , my understanding is that Talos should upgrade upon receiving a configuration with an installer pointing to something diffrent from the current version and reboot. This is not what I am witnessing.

machine:
  install:
    image: factory.talos.dev/installer/e048aaf4461ff9f9576c9a42f760f2fef566559bd4933f322853ac291e46f238:v1.7.0
    wipe: true

I can see in the dashboard, that the patch is being applied

 apply config request: mode auto(no_reboot)

and can confirm the config is there

talosctl --talosconfig talosconfig get machineconfig -o yaml -n 116.202.100.70|grep installer -B4 -A4
            disableManifestsDirectory: true
        network: {}
        install:
            disk: /dev/sda
            image: factory.talos.dev/installer/e048aaf4461ff9f9576c9a42f760f2fef566559bd4933f322853ac291e46f238:v1.7.0
            wipe: true

But no upgrade is done. Not even after reboot. And upgrade using the talosctl works well. Is it me not reading the documentation correctly, an issue in the documentation or a bug?

This is important for me to understand whether a reboot would happen in case we generate an new factory image in our CI using the Terraform module.

Thx!

Client: Tag: v1.7.7 SHA: 68794084 Built: Go version: go1.22.7 OS/Arch: linux/amd64 Server: NODE: 116.202.100.70 Tag: v1.7.7 SHA: 68794084 Built: Go version: go1.22.7 OS/Arch: linux/amd64 Enabled: RBAC

smira commented 1 week ago

No, this is not the case.

.machine.install section is only used for initial install of Talos, and ignored otherwise.

spastorclovr commented 1 week ago

Thanx very much @smira I misinterpreted this bit from the documentation

When it’s time to upgrade Talos, generate a custom installer container for a new version of Talos, push it to a registry, and perform upgrade pointing to the custom installer image.

You might wanna add "preform a manual upgrade" to make it clearer (if it is not enough already) .

Thx again!

spastorclovr commented 1 week ago

Sorry to bother you @smira on what could be only some misunderstanding from my part but when trying to boot from a 1.7.5 version ISO , passing as user data (I am testing this on Hetzner) the controlplane manifest including the installer pointing to a custom image generated from the image factory, shouldn't I see the server

smira commented 1 week ago

yes, that should be it, you can inspect the logs yourself and see what happened.

spastorclovr commented 1 week ago

dmesg.txt In that case this is not what I am experiencing. Once booted, the machineconfig is well-retrieved, I can check it is applied successfully but no reboot is performed and no obvious-related error in the dmesg

The manifest passed as user data is the one generated by the talosctl gen config with only the install line changed to image: factory.talos.dev/installer/e048aaf4461ff9f9576c9a42f760f2fef566559bd4933f322853ac291e46f238:v1.7.0

Pease find in attachment the dmesg from the machine. Let me know if you need anything else.

Here is the only bit changed int the controlplane.yaml

 machine:

    install:
        disk: /dev/sda # The disk used for installations.
        image: factory.talos.dev/installer/e048aaf4461ff9f9576c9a42f760f2fef566559bd4933f322853ac291e46f238:v1.8.0
        wipe: false # Indicates if the installation disk should be wiped at installation time.
spastorclovr commented 1 week ago

Could this issue be reopen @smira ? Let me know if you need anything else.

smira commented 1 week ago

Your machine is already installed to the disk, and seems to be booting from the disk (that's from your dmesg.txt).

So no install would ever happen.

spastorclovr commented 1 week ago

Very true .. thanks for checking and replying to me.

I wrongly believed that the image passed when creating hetzner servers was a ISO-type image where the server would boot from then perform the install. It is instead a disk image, so yes no install is performed at all ... :roll_eyes:

Thanks again for taking the time and sorry for the noise.