kairos-io / kairos

:penguin: The immutable Linux meta-distribution for edge Kubernetes.
https://kairos.io
Apache License 2.0
1.13k stars 97 forks source link

Create a kairos "installer" #2968

Open jimmykarily opened 2 weeks ago

jimmykarily commented 2 weeks ago

Right now, the way to create a Kairos "derivative" image is to either feed a base image to our dockerfiles of take one of our images and use it as a base image for another dockerfile.

Our dockerfiles are rather complex and the logic very hard to follow. This makes them very difficult to extend and test. Also, using dockerfiles, makes docker a hard dependency (or some other tool that can build dockerfiles anyway).

We could move all the dockerfile logic in a go binary (let's call it the "installer" for now but we need a better name to avoid confusion with the agent that installs Kairos on the machines). By doing that, one would simply run the binary inside an OS and all the dependencies to make it a Kairos OS would be installed with no additional dependencies.

bencorrado commented 4 days ago

I would love to see a factory provisioning step added for this, ideally this would be a yip/cloudconfig file that is loaded at installation to augment the installation process that is not persisted to the system. I have gotten called out on security audits for leaving installation scripts on deployed devices as it gives more information than is needed to a potential hacker about how the system is bootstrapped.

Currently we run things in

  kairos-uki-install.after:
    - commands:
      - |

which allow us to add unique keys, names, capture device identities, etc.

and then at the end of that stage, before the first boot, we actually remove that whole block like this:

  # Cleanup unwanted stages from YAML file
  echo "Cleaning up the bootstrapping instructions"
  /oem/yq eval -i 'del(.stages."kairos-uki-install.after")' /oem/90_custom.yaml
  rm /oem/yq

It works, but it feels crude and fragile. If I could just add another cloud-config to the install process (we use a PiKVM to mount a unique disks to each device, one the Kairos installer, the other the user-data with the yip/cloud-config) It could be nice to be able to use an installer-data file too.