rancher / elemental-toolkit

:snowflake: The toolkit to build, ship and maintain cloud-init driven Linux derivatives based on container images
https://rancher.github.io/elemental-toolkit/docs/
Apache License 2.0
290 stars 49 forks source link

Elemental on Raspberry Pi Fails OEM/Persistence Reset due to Empty `/oem` Directory #2116

Open atoy3731 opened 3 months ago

atoy3731 commented 3 months ago

elemental-toolkit version: elemental-operator-1.5.3 (via Rancher UI Extension)

CPU architecture, OS, and Version: Raspberry Pi 4B - 8GB

Describe the bug When "repurposing" an Elemental Raspberry Pi node, the reset process fails and is unable to re-register to Rancher to be utilized again. The elemental-register-reset.service service on the Pi has 2 main errors:

Initially:

failed installing grub: open /sys/firmware/efi/efivars/Boot0000-8be4df61-93ca-11d2-aa0d-00c098032bc: read-only file system

Followed repeated by:

initializing configuration: merging config: open /oem/registration/config.yaml: no such file or directory

To Reproduce

  1. In Rancher, create a new Registry Endpoint using this configuration (potentially modifying your device):

    config:
      cloud-config:
        users:
          - name: root
            passwd: root
      elemental:
        install:
          debug: true
          device: /dev/mmcblk0
          disable-boot-entry: true
          poweroff: false
          reboot: true
          snapshotter:
            type: loopdevice
        registration:
          auth: tpm
          emulate-tpm: true
          emulated-tpm-seed: -1
        reset:
          enabled: true
          reboot: true
          reset-oem: true
          reset-persistent: true
    machineInventoryLabels:
      type: raspberry-pi
  2. On a Raspberry Pi device, ensure you have terminal access (via a monitor and keyboard) and follow the Elemental documentation to produce a raw image, specifically the Preparing the seed image (aarch64) manually section.

  3. Once the raw image is produced, burn it to a USB flash drive and plug it into a Raspberry Pi 4B with an empty SD card. After boot and install, the Raspberry Pi should show up as an active Machine in Rancher's Elemental UI.

  4. Create an Elemental Kubernetes cluster with a single node that matches the Machine's labels (type=raspberry-pi)

  5. Once the cluster is healthy, delete the cluster.

  6. Wait for the Raspberry Pi to restart via the monitor, when it restarts, check the logs via journalctl -u elemental-register-reset.service

Expected behavior Raspberry Pi Elemental node rejoins and becomes active in Rancher after OEM/Persistence reset.

Logs IMG_3075 IMG_3074

Additional context N/A

davidcassany commented 2 months ago

I have been looking at it and I think this was already fixed in head but not backported. Gonna try to validate backporting it to v2.1.x series fixes the issue.