:mega: Kairos release v3.1.1

[mudler]

🗺 What's left for release

• [x] https://github.com/kairos-io/kairos/issues/2670
• [x] https://github.com/kairos-io/kairos/issues/2682
• [x] https://github.com/kairos-io/kairos/issues/2664
• [x] https://github.com/kairos-io/kairos/issues/2707
• [x] https://github.com/kairos-io/kairos/issues/2708
• [x] https://github.com/kairos-io/kairos/issues/2665
• [x] https://github.com/kairos-io/kairos/issues/2715
• [x] https://github.com/kairos-io/kairos/issues/2727

See also: https://github.com/kairos-io/kairos/pull/2705#issuecomment-2217790248

🔦 Highlights

This release is a patch release to address CVEs and to fix known issues with #1792 .

✅ Release Checklist

• [x] Stage 0 - Finishing Touches
  • [x] Check if Kairos-docs has any open PRs that need to be merged
  • [x] Check if osbuilder is in the wanted version/latest
  • [x] Check if any kairos/packages were bumped. Ensure they were merged and repo updated (https://github.com/kairos-io/kairos-framework/issues/2)
    • [x] Cut a new release of the kairos-framework images
    • [x] Bump the kairos-framework image in kairos
  • [x] CI tests are passing
  • [x] There are no critical CVEs in our internal components
  • [x] Consider cutting an rc, alpha, ... based on changes on the CI
• [x] Stage 1 - Manual testing
  • How: Using the assets from master, make sure that test scenarios not covered by automatic tests are passing, and that docs are still aligned
  • [x] Fedora flavor install, and manual upgrade works
  • [x] Any flavor interactive install
  • [x] Any flavor recovery reset
  • [x] Any flavor k3s
  • [x] ARM images (Fedora, Alpine) standard manual testing (install, boot, recovery, reset, passive, k3s, etc...) @Itxaka
  • [x] ARM images (openSUSE, alpine) boots and manual upgrade works
  • [x] ARM images passive and recovery booting
  • [x] ARM images reset works
  • [x] ARM images /oem exists
• [x] Stage 3 - Release
  • [x] Tag the release on master
  • [x] Update the release with any known issues
• [x] Stage 4 - Announcement
  • [x] Merge docs updates for kairos and k3s version updates
  • [x] Create a branch vX.Y.Z on the docs (not tagging), so the new release can be built and displayed on the menu. Ideally open a PR so we can review and add/remove some commits if needed (in case we have documented WIP which is not available on the given release)
  • [x] Blog post announcement

[Itxaka]

Arm Alpine RPI testing

• Using iso built from master, kairos-alpine-3.19-standard-arm64-rpi4-v3.1.0-20-g3e1b056-k3sv1.30.2
• Using USB as media install on RPI4

Checks:

• Boots from USB media :heavy_check_mark:

• Shows all entries :heavy_check_mark:

• Expands PERSISTENT fs on first boot :heavy_check_mark:

• Can login via ssh :heavy_check_mark:

• User is not required to change password on next login (issue #2439 and #1994 ) :heavy_check_mark:

• K3S starts with no issues (so cgroups are ok, issue #2002 ) :heavy_check_mark:

  localhost:~$ sudo k3s kubectl get nodes
  NAME        STATUS   ROLES                  AGE    VERSION
  localhost   Ready    control-plane,master   2m7s   v1.30.2+k3s1
  localhost:~# k3s kubectl get pods --all-namespaces
  NAMESPACE     NAME                                      READY   STATUS      RESTARTS   AGE
  kube-system   coredns-576bfc4dc7-lg69t                  1/1     Running     0          5m25s
  kube-system   helm-install-traefik-7l8bp                0/1     Completed   1          5m25s
  kube-system   helm-install-traefik-crd-jkrb2            0/1     Completed   0          5m25s
  kube-system   local-path-provisioner-86f46b7bf7-2rqpn   1/1     Running     0          5m25s
  kube-system   metrics-server-557ff575fb-fm7xk           1/1     Running     0          5m25s
  kube-system   svclb-traefik-18b987e0-wx4z4              2/2     Running     0          4m56s
  kube-system   traefik-5fb479b77-cxppp                   1/1     Running     0          4m56s

• Can boot on passive :heavy_check_mark:

• Can boot on recovery :heavy_check_mark:

• On recovery, COS_PERSISTENT is not mounted :heavy_check_mark:

• Autoreset works :heavy_check_mark:

  2006-08-14T08:35:31Z INF Running stage: before-reset
  2006-08-14T08:35:31Z INF Done executing stage 'before-reset'
  2006-08-14T08:35:31Z INF Running stage: before-reset.after
  2006-08-14T08:35:31Z INF Done executing stage 'before-reset.after'
  2006-08-14T08:35:31Z INF Copying /run/initramfs/cos-state/cOS/recovery.img source to /run/cos/state/cOS/active.img
  2024-07-19T08:17:29Z INF Finished copying /run/initramfs/cos-state/cOS/recovery.img into /run/cos/state/cOS/active.img
  2024-07-19T08:17:31Z INF Using grub config dir /run/cos/active/etc/cos/grub.cfg
  2024-07-19T08:17:31Z INF Copying grub contents from /run/cos/active/etc/cos/grub.cfg to /run/cos/state/grub2/grub.cfg
  2024-07-19T08:17:31Z INF Generating grub files for efi on /dev/sda
  2024-07-19T08:17:32Z WRN did not find grub font ascii.pf2 under /run/cos/active
  2024-07-19T08:17:33Z WRN did not find grub font euro.pf2 under /run/cos/active
  2024-07-19T08:17:33Z WRN did not find grub font unicode.pf2 under /run/cos/active
  2024-07-19T08:17:33Z INF Running after-reset-chroot hook
  2024-07-19T08:17:33Z INF Running stage: after-reset-chroot.before
  2024-07-19T08:17:33Z INF Done executing stage 'after-reset-chroot.before'
  2024-07-19T08:17:33Z INF Running stage: after-reset-chroot
  2024-07-19T08:17:33Z INF Done executing stage 'after-reset-chroot'
  2024-07-19T08:17:33Z INF Running stage: after-reset-chroot.after
  2024-07-19T08:17:33Z INF Done executing stage 'after-reset-chroot.after'
  2024-07-19T08:17:33Z INF Running stage: after-reset-chroot.before
  2024-07-19T08:17:33Z INF Done executing stage 'after-reset-chroot.before'
  2024-07-19T08:17:33Z INF Running stage: after-reset-chroot
  2024-07-19T08:17:33Z INF Done executing stage 'after-reset-chroot'
  2024-07-19T08:17:33Z INF Running stage: after-reset-chroot.after
  2024-07-19T08:17:33Z INF Done executing stage 'after-reset-chroot.after'
  2024-07-19T08:17:33Z INF Setting default grub entry to Kairos
  2024-07-19T08:17:33Z INF Copying /run/cos/state/cOS/active.img source to /run/cos/state/cOS/passive.img
  2024-07-19T08:17:41Z INF Finished copying /run/cos/state/cOS/active.img into /run/cos/state/cOS/passive.img
  2024-07-19T08:17:49Z INF Running after-reset hook
  2024-07-19T08:17:49Z INF Running stage: after-reset.before
  2024-07-19T08:17:49Z INF Done executing stage 'after-reset.before'
  2024-07-19T08:17:49Z INF Running stage: after-reset
  2024-07-19T08:17:49Z INF Processing stage step 'Mount state'. ( commands: 1, files: 0, ... )
  2024-07-19T08:17:49Z INF Command output: 
  2024-07-19T08:17:49Z INF Processing stage step 'Hook boot assessment grub configuration'. ( commands: 1, files: 0, ... )
  2024-07-19T08:17:49Z INF Command output: 
  2024-07-19T08:17:49Z INF Processing stage step 'Grub branding'. ( commands: 1, files: 0, ... )
  2024-07-19T08:17:49Z INF Command output: '/etc/kairos/branding/grubmenu.cfg' -> '/tmp/mnt/STATE/grubmenu'
  2024-07-19T08:17:49Z INF Processing stage step 'Add boot assessment grub configuration'. ( commands: 0, files: 1, ... )
  2024-07-19T08:17:49Z INF Processing stage step 'umount state'. ( commands: 1, files: 0, ... )
  2024-07-19T08:17:49Z INF Command output: 
  2024-07-19T08:17:49Z INF Done executing stage 'after-reset'
  2024-07-19T08:17:49Z INF Running stage: after-reset.after
  2024-07-19T08:17:49Z INF Done executing stage 'after-reset.after'
  2024-07-19T08:17:49Z INF Running stage: after-reset.before
  2024-07-19T08:17:49Z INF Done executing stage 'after-reset.before'
  2024-07-19T08:17:49Z INF Running stage: after-reset
  2024-07-19T08:17:49Z INF Done executing stage 'after-reset'
  2024-07-19T08:17:49Z INF Running stage: after-reset.after
  2024-07-19T08:17:49Z INF Done executing stage 'after-reset.after'
  2024-07-19T08:17:49Z INF Unmounting disk partitions
  2024-07-19T08:17:52Z INF Starting rsync...
  2024-07-19T08:17:52Z INF Finished syncing

• After reset, PERSISTENT is clean and system behaves as just installed :heavy_check_mark:

  localhost:~$ sudo k3s kubectl get nodes
  NAME        STATUS   ROLES                  AGE   VERSION
  localhost   Ready    control-plane,master   8s    v1.30.2+k3s1

• Upgrade to quay.io/kairos/alpine:3.19-core-arm64-rpi4-master

• First time failed becuase the ntp service was not ready yet, so system date was too old :x:

• Waiting a few seconds to ftpd to set the system date, then triggered it and all was ok :heavy_check_mark:

  • Upgrade done thrice, once from recovery, once from active, one from passive :heavy_check_mark:
    • From recovery
    • Active was moved to passive :heavy_check_mark:
    • Upgraded image was moved to active :heavy_check_mark:
    • Booting from active shows that the image is now a base image, so upgrade went ok :heavy_check_mark:
    • From Active
    • Active was moved to passive :heavy_check_mark:
    • Upgraded image was moved to active :heavy_check_mark:
    • Booting from active shows that the image is now a base image, so upgrade went ok :heavy_check_mark:
    • From Passive
    • Passive was left alone :heavy_check_mark:
    • Upgraded image was moved to active :heavy_check_mark:
    • Booting from active shows that the image is now a base image, so upgrade went ok :heavy_check_mark:

Fedora 40 ARM rpi image checks: Complete and good to go :heavy_check_mark:

Ntpd stuff should be commented in the release notes and we should try to make it run earlier in the process so when login is presented the date is ready. Currently having to wait up to 30 seconds is not a good thing :exclamation:

[Itxaka]

Arm Fedora RPI testing

Not supported. Dockerfiles need changes to support separated packages for arm64 VS amd64

For future releases: https://github.com/kairos-io/kairos/pull/2748