:sparkles: iPXE installation via serial console on Equinix Metal

[mauromorales]

With Equinix Metal you only get serial access to the console of the physical server. Hence, to deploy Kairos there over iPXE and have working console access, a couple of changes are needed:

• In the .ipxe boot file used to boot the node, we need to set the console=tty0 console=ttyS1,115200n8 dracut parameter
• In the rootfs.squashfs file for Kairos, we need to edit the /etc/cos/bootargs.cfg file and modify the console= options to match those used in the .ipxe file

This allows Kairos (openSUSE) to iPXE boot with serial access, as well as have working serial access upon the next boot from local HDD.

However, to install Kubernetes onto it, it downloads a new container OS image (e.g. Ubuntu), which overwrites /etc/cos/bootargs.cfg with its own version. This typically has boot options that are not compatible with serial access. Typically it’s wrong console= options and sometimes there’s also a vga=795 option in there that breaks serial access. We would like to find a way to manage the bootargs across OS images, so that we can add, edit and delete boot arguments as needed, without requiring the building of custom OS images.

Original request slack link

[mudler]

this is currently the options we have:

• You can set the tty during installation , so the grub config will have the tty specified in the config To set the tty during install you could rely on elemental and set the following in the config file:

  options:
  tty: yourTTY

• You can pass additional cmdline args via the config file, that will be loaded automatically in the OS

  # The install block is to drive automatic installations without user interaction.
  install:
  
  # Set grub options
  grub_options:
  # additional Kernel option cmdline to apply
  extra_cmdline: "config_url=http://"
  # Same, just for active
  extra_active_cmdline: ""
  # Same, just for passive
  extra_passive_cmdline: ""
  # Change GRUB menu entry
  default_menu_entry: ""
  # Environmental variable to set to the installer calls

• Finally, the base cmdline of the OS is defined in /etc/cos/bootargs.cfg and that's the default of the Kairos derivative that you are using to to have proper defaults. It is the responsability of the derivative to have proper defaults.

Can you try to set up the tty during installation by setting the options? does that work for you?

options:
  tty: ...

[mudler]

This sounds more as a spike for now - there is no clear action path, and there is no real bug hiding here, it's just how we setup the node.

[mudler]

One solution I see is:

we could have less defaults in the base image, and actually treat them as extra injected args if not provided by the user. That is, we don't specify anymore console=* options in the bootargs.cfg of Kairos, but instead the agent should detect if the user didn't provided any, and inject that in the oem configuration of the system (and during install).