rancher / k3os

Purpose-built OS for Kubernetes, fully managed by Kubernetes.
https://k3os.io
Apache License 2.0
3.5k stars 397 forks source link

Auto-upgrade-controller should not downgrade if pre-release is installed #719

Closed MarkusGeigerDev closed 2 years ago

MarkusGeigerDev commented 3 years ago

Version (k3OS / kernel) k3os version v0.21.1-k3s1r0

Architecture Raspberry Pi 3B+

Describe the bug If a node has the k3os.io/upgrade=enabled label set, the auto-upgrade controller always applies https://github.com/rancher/k3os/releases/latest, even if the current release is newer (i.e. a pre-release), therefore effectively downgrading the system. This should not happen.

To Reproduce

Expected behavior The auto-update controller should ignore latest versions that are earlier than the current version

Actual behavior The auto-update controller always applies latest, even if that results in a downgrade

dweomer commented 2 years ago

The k3os-latest Plan is an outstanding intent to mutate your cluster subject to, by default, resolution of the "latest" tag k3OS tag on github. If you would rather avoid the side-effects of unintended mutations I encourage you to:

  1. make sure none of your node labels qualify for the k3os-latest node-selector (it will get redeployed on upgrade unless you take steps to delete it from your server nodes before k3s starts. if the redeployed manifest has a timestamp newer than your latest edit it will most likely get overwritten)
  2. install a plan that you control which does not point to a channel not under your control (pin it to a version or consider using a channel server such as we do for k3s, i.e. https://update.k3s.io/v1-release/channels)

Additionally, the SUC does not attempt to determine if a version is newer than what is currently installed, only that it is different. Another good reason to orchestrate upgrades with a plan that you control :smiley: (rancher-orchestrated upgrades for k3s and rke2 do exactly this but, sadly, k3OS upgrades are not supported)