Examples showing `#cloud-config` files with `stages.boot` sections don't work

Kairos version:

PRETTY_NAME="Ubuntu 24.04 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
KAIROS_VERSION="v3.1.3-v1.31.0-k3s1"
KAIROS_PRETTY_NAME="kairos-standard-ubuntu-24.04 v3.1.3-v1.31.0-k3s1"
KAIROS_FLAVOR="ubuntu"
KAIROS_RELEASE="v3.1.3"
KAIROS_REGISTRY_AND_ORG="quay.io/kairos"
KAIROS_BUG_REPORT_URL="https://github.com/kairos-io/kairos/issues"
KAIROS_GITHUB_REPO="kairos-io/kairos"
KAIROS_NAME="kairos-standard-ubuntu-24.04"
KAIROS_VERSION_ID="v3.1.3-v1.31.0-k3s1"
KAIROS_IMAGE_LABEL="24.04-standard-amd64-generic-v3.1.3-k3sv1.31.0-k3s1"
KAIROS_ARTIFACT="kairos-ubuntu-24.04-standard-amd64-generic-v3.1.3-k3sv1.31.0+k3s1"
KAIROS_FLAVOR_RELEASE="24.04"
KAIROS_SOFTWARE_VERSION_PREFIX="k3s"
KAIROS_ID="kairos"
KAIROS_IMAGE_REPO="quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.1.3-k3sv1.31.0-k3s1"
KAIROS_VARIANT="standard"
KAIROS_MODEL="generic"
KAIROS_TARGETARCH="amd64"
KAIROS_HOME_URL="https://github.com/kairos-io/kairos"
KAIROS_SOFTWARE_VERSION="v1.31.0+k3s1"
KAIROS_ID_LIKE="kairos-standard-ubuntu-24.04"
KAIROS_FAMILY="ubuntu"

CPU architecture, OS, and Version:

Linux beta 6.8.0-44-generic #44-Ubuntu SMP PREEMPT_DYNAMIC Tue Aug 13 13:35:26 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug

After trying to use the examples that use PXE and cloud-init to provision a system, none of the steps in stages: boot: were being applied. If I rename the step to boot.after: it will work.

After a lot of digging through the code, it appears that Yip detects the "#cloud-config" line at the top of the file and transpiles from cloudInit style to yipYaml which tries to merge the stages: sections for boot. I'm not sure if this is a problem when the PXE install happens or after the installation occurs.

To Reproduce

Create cloud-config file with "#cloud-config" at the top, auto install indicated, and "stages: boot:" section that does something that can be observed (e.g., touch a file in an ephemeral, per-boot directory)
PXE boot client with config_url= set to that cloud-config

Expected behavior

Installation should occur, reboot, and then apply steps from "stages: boot:"

Logs

Additional context

When the steps are renamed to "boot.after" this appears to work.

umm, I tested this with the following cc:

#cloud-config

install:
    grub_options:
        extra_cmdline: rd.immucore.debug
    poweroff: false
    reboot: false
stages:
    boot:
        - files:
            - content: "test\n"
              group: 1000
              owner: 1000
              path: /run/bar
              permissions: 511
          name: BOOT STAGE TEST
    initramfs:
        - name: Set user and password
          users:
            kairos:
                passwd: kairos
        - hostname: kairos-{{ trunc 4 .Random }}

Installed, rebooted and I can see the file there:

kairos@kairos-xhwk:~$ stat /run/bar 
  File: /run/bar
  Size: 25          Blocks: 8          IO Block: 4096   regular file
Device: 0,26    Inode: 2335        Links: 1
Access: (0777/-rwxrwxrwx)  Uid: ( 1000/  ubuntu)   Gid: ( 1000/  ubuntu)
Access: 2024-10-02 10:11:38.036511722 +0000
Modify: 2024-10-02 10:11:38.036511722 +0000
Change: 2024-10-02 10:11:38.036511722 +0000
 Birth: 2024-10-02 10:11:38.036511722 +0000
kairos@kairos-xhwk:~$ cat /run/bar 
test

And I can see the stage being run in the service:

Oct 02 10:11:38 kairos-xhwk kairos-agent[1695]: 2024-10-02T10:11:38Z INF Processing stage step ''. ( commands: 0, files: 0, ... )
Oct 02 10:11:38 kairos-xhwk kairos-agent[1695]: 2024-10-02T10:11:38Z INF Processing stage step 'BOOT STAGE TEST'. ( commands: 0, files: 1, ... )
Oct 02 10:11:38 kairos-xhwk kairos-agent[1695]: 2024-10-02T10:11:38Z INF Done executing stage 'boot'

And this is with an example from the examples docs you linked. The only things changed were to add a name to the stage so I could see it on the logs and setting the group/owner to 1000 so it picks up the default ubuntu user on my iso.

If you want us to solve this we will need:

full cc you used
logs from the boot stage service (journalctl -u cos-setup-boot)

Also can you confirm that this happens AFTER install, i.e. from an installed system? During installation (livecd/pxe) im not sure its possible for the boot service to read from the config_url as that is ingested after it has run, so it would be good to clear that the reproduction is as follow:

Cloud Config has some stages.boot actions (create file, set hostname, whatever)
Boot from IPXE with config_url
Install
Reboot to installed system
Boot service runs but the boot stages are not executed
Moving those to boot.after makes them run

Thanks!

I'm closing this since because it's stalled. Feel free to re-open if you have more information to help us reproduce.

kairos-io / kairos

Examples showing `#cloud-config` files with `stages.boot` sections don't work #2892