Unable to iPXE boot on bare metal using netboot.xyz

remedywilliams commented 1 year ago

Description

Unable to iPXE boot flatcar (stable channel)

Impact

Inability to boot Flatcar Linux for install to bare metal.

Environment and steps to reproduce

Setup
- Hardware
- pfsense using itself as tftp/next server
- HP EliteDesk 800 G2
  - Legacy Boot Disabled (forcing UEFI)
  - tested with TPM on/off
  - booted into netboot.xyz via IPv4
Task
- (within netboot.xyz)
- Linux Network Installs (64-bit) > Flatcar Container Linux > Flatcar Container Linux Stable Channel
- set ignition.config.url with ignition config

3/4. Action(s) / Error:

while booting, the following was noticed [0.044319] localhost kernel: Kernel command line: rootflags=rw mount.usrflags=ro flatcar_production_pxe.vmlinuz flatcar.first_boot=1 ignition.config.url=OMITTED flatcar.autologin=tty1 flatcar.autologin=ttyS0 initrd=flatcar_production_pxe_image.cpio.gz [0.483890] localhost kernel: fail to initialize ptp_kvm [0.689238] localhost kernel: Trying to unpack rootfs image as initramfs... [0.689268] localhost kernel: Initramfs unpacking failed: invalid magic at start of compressed archive [91.074440] localhost systemd[1]: dev-disk-by\x2dlabel-OEM.device: Job dev-disk-by\x2dlabel-OEM.device/start timed out. [91.082245] localhost systemd[1]: Timed out waiting for device /dev/disk/by-label/OEM. [91.089188] localhost systemd[1]: Dependency failed for Ignition (setup). [91.095350] localhost systemd[1]: Dependency failed for Ignition (files). [91.097851] localhost systemd[1]: Dependency failed for Ignition Complete. [91.100194] localhost systemd[1]: Dependency failed for Initrd Default Target. [91.102620] localhost systemd[1]: initrd.target: Job initrd.target/start failed with result 'dependency'. [91.105190] localhost systemd[1]: initrd.target: Triggering OnFailure= dependencies.

In between 0.6 and 91 seconds is a hanging message, switching between (1 of 2) A start job is running for /dev/disk/by-label/OEM (58s / 1min 30s) (2 of 2) A start job is running for /dev/disk/by-partlabel/USR-A (59s / 1min 30s)

Expected behavior

Ability to install FCL via iPXE on bare metal using netbook.xyz

pothos commented 1 year ago

netboot.xyz is broken for BIOS boot (see https://github.com/flatcar/Flatcar/issues/586) but UEFI was reported to work. I rather recommend you to set the iPXE script up yourself: https://www.flatcar.org/docs/latest/installing/bare-metal/booting-with-ipxe/#setting-up-the-boot-script

#!ipxe
set base-url http://stable.release.flatcar-linux.net/amd64-usr/current
kernel ${base-url}/flatcar_production_pxe.vmlinuz initrd=flatcar_production_pxe_image.cpio.gz flatcar.first_boot=1 ignition.config.url=https://example.com/pxe-config.ign
initrd ${base-url}/flatcar_production_pxe_image.cpio.gz
boot

One more example is https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_packet.ipxe where it makes use of relative paths. Make sure you use the right serial console argument in case you don't see initrd boot logs.

jepio commented 1 year ago

A different version of netboot.xyz might be needed.

remedywilliams commented 1 year ago

@pothos thank you for that additional info. Yes I'm aware of the BIOS bug (I thought I referenced it in the desc but I don't see it), which is why I ensured disabling Legacy Boot on the machine, yet interestingly I still noticed the invalid magic message.

I've since booted using bare ipxe, so it may be as @jepio mentioned that netboot.xyz may need to address this.

pothos commented 1 year ago

When switching to PXE, the default PXE mode is BIOS and you have to explicitly tell to use the PXE UEFI mode, e.g., when using ipmitool.

flatcar / Flatcar