unable to detect active ESP

[firasuke]

Hey there,

I am attempting to create a live installation ISO for a custom distribution I am making. I have a rootfs setup, with a bootloader configured (syslinux for mbr/bios, and grub for EFI). I am using Arch Linux kernel and modules (since booster works just fine with Arch), and I am using booster universal to generate an initramfs image for my custom distribution.

Regardless of the mode I boot in under QEMU, and regardless of what I use syslinux vs GRUB I am receiving the following error:

booster: unable to detect active ESP: open /sys/firmware/efi/efivars/LoaderDevicePartUUID-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f: no such file or directory

I have stumbled upon the following issue #149 which seems similar, and a patch was proposed their to remove this check (as my custom distribution is close in structure to Alpine, so it made sense). After trying the patch I am left with the following error booster: /dev/sr0 unknown drive type which might be related to the device manager I am using mdevd, but then again shouldn't booster run before any service/device manager has been started? or should there be hooks for the device manager like udev?

It seems that booster is trying to load efivarfs which is working, but then is trying to open a non-existent directory? I thought booster handled the mounting of sys? If so why is it failing? And why is it trying to detect an ESP when in MBR/BIOS mode? Also, could it be relying on the system's service boot scripts to mount sys or for more functionality? If so, shouldn't booster not do that because it comes before in the boot graph order?

[anatol]

Thanks for the report. I am happy to debug this problem. Is there a way to reproduce the failing case with QEMU? I am using Arch and QEMU+failing image would be the best option I think.

Or another option is to enable debug logs with booster.log=debug,console and then post its output here.

[firasuke]

Alright I added booster.log=debug,console to kernel cmdline and I got the following additional information:

root= param is not specified. Use GPT partition autodiscovery with guid type 4f68bce3-e8cd-4db1-96e7-fbcaf984b709

booster: unable to detect active ESP: open /sys/firmware/efi/efivars/LoaderDevicePartUUID-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f: no such file or directory

[anatol]

And what is the debug output in case of booster: /dev/sr0 unknown drive type error?

[anatol]

So it looks like you do not have a root= boot param, so booster switches into GPT autodiscovery mode. Is there a reason the bootloader (syslinux and grub) do not specify root= param?

[firasuke]

Here's what I have in my grub.cfg (using GRUB for EFI):

...
    linux  /boot/vmlinuz modules=loop,squashfs,erofs rw booster.log=debug,console
    initrd /boot/initramfs
...

and in my syslinux.cfg (using ISOLINUX for BIOS):

    KERNEL /boot/vmlinuz
    INITRD /boot/initramfs
    APPEND rw booster.log=debug,console

[anatol]

So you have 2 options here

• specify root=/dev/XXX
• rely on GPT autodiscovery. And you need the efi module loaded and fully functional.

The later case won't work for non-EFI configuration though.

[firasuke]

So you have 2 options here

* specify `root=/dev/XXX`

* rely on GPT autodiscovery. And you need the `efi` module loaded and fully functional.

The later case won't work for non-EFI configuration though.

In the case of an ISO, the root filesystem will be stored mostly on /dev/sr0, so I passed root=/dev/sr0 and booster throws a huge log, with udev doing detections and stuff, and then it throws:

...
booster: /dev/sr0 unknown drive type
...

should i try adding rootfstype? I am also pretty sure the loop module is being added to booster..

[anatol]

Please add root=/dev/sr0 and the debug parameters. And then post the debug logs.

[firasuke]

I am unable to capture the output of QEMU even with -nographics and with -D or -serial...

[anatol]

Try -nographic -display none maybe?

[anatol]

Also do you have a reproduction insructions for the image you are making? Ideally all the cases that can break need to be part of booster's integration tests here https://github.com/anatol/booster/tree/master/tests.

[firasuke]

Try -nographic -display none maybe?

Nope, that did not work... I will upload screenshots showcasing most of the error log.

Also do you have a reproduction insructions for the image you are making? Ideally all the cases that can break need to be part of booster's integration tests here https://github.com/anatol/booster/tree/master/tests.

I am creating an ISO for a custom Linux distribution I am working on. I have a root filesystem consisting mainly of musl libc and toybox for user space and some other utilities.

I have a boot directory where I copy Arch's kernel as vmlinuz, and its modules to usr/lib/modules.

I am then creating a universal booster initramfs image by running:

booster \
    build \
    --force \
    --compression=zstd \
    --config=booster.yaml \
    --universal \
    --strip \
    rootfs/boot/initramfs

Here's what I have inside booster.yaml

network:
  dhcp: on
universal: true
modules: loop,squashfs,erofs
compression: zstd
strip: true

Here's what I have inside boot/grub/grub.cfg:

...
    linux  /boot/vmlinuz modules=loop,squashfs,erofs rw booster.log=debug,console
    initrd /boot/initramfs
...

Here's what I have inside boot/syslinux/syslinux.cfg:

    KERNEL /boot/vmlinuz
    INITRD /boot/initramfs
    APPEND rw booster.log=debug,console

I am using the following instructions to create the ISO:

grub-mkrescue -o mycustom.iso rootfs

I also use xorriso:

xorriso -as mkisofs \
    -iso-level 3 \
    -full-iso9660-filenames \
    -rational-rock \
    -eltorito-boot boot/syslinux/isolinux.bin \
    -eltorito-catalog boot/syslinux/boot.cat \
    -no-emul-boot -boot-load-size 4 -boot-info-table \
    -isohybrid-mbr rootfs/boot/syslinux/isohdpfx.bin \
    -output mycustom.iso rootfs/

When I run the ISO, here's the error I get in EFI mode:

qemu-system-x86_64 -enable-kvm -cpu host -boot d -cdrom mycustom.iso -m 1G -bios /usr/share/ovmf/x64/OVMF.fd

and in regular BIOS/mbr mode:

qemu-system-x86_64 -enable-kvm -cpu host -boot d -cdrom mycustom.iso -m 1G

[firasuke]

It seems the error is as follows:

found a new device /dev/sr0
blkinfo for /dev/sr0: type=mbr UUID=00000000 LABEL=
booster: /dev/sr0 unknown drive type
/dev/sr0 unknown drive type

Apparently it is failing to detect /dev/sr0 correctly, as it is of type iso9660 and not mbr? Also is there a way to specify a certain LABEL or UUID (perhaps the same one that the ISO was created with -volid to make it easier to auto-detect it for this use case?

[anatol]

Apparently it is failing to detect /dev/sr0 correctly, as it is of type iso9660 and not mbr?

Indeed, this one of the problems.

Another problem is that SMART tries to read data from /dev/sr0 and fails and it is where the unknown drive type comes from.

Thank you for the provided instructions. I added an integration test for .iso that show the error. And handled the error case. The changes are pushed to issue-223 branch. Please pull it, compile the init, and pack it with your .iso. Let me know the results.

[firasuke]

Apparently it is failing to detect /dev/sr0 correctly, as it is of type iso9660 and not mbr?

Indeed, this one of the problems.

Another problem is that SMART tries to read data from /dev/sr0 and fails and it is where the unknown drive type comes from.

Thank you for the provided instructions. I added an integration test for .iso that show the error. And handled the error case. The changes are pushed to issue-223 branch. Please pull it, compile the init, and pack it with your .iso. Let me know the results.

Alright, thanks for your work on this.

I tested the new branch, and unfortunately with EFI (grub) it is still showing the same error as above, but with syslinux (mbr/BIOS mode) it is failing but with some difference in the error log:

found a new device /dev/sr0
blkinfo for /dev/sr0: type=mbr UUID=44523d4f LABEL=
booster: /dev/sr0 unknown drive type
/dev/sr0 unknown drive type

Again is mbr the correct filesystem/type for this? Also shouldn't it be mounted somewhere (/new_root) before being switch_rooted into?

A friend of mine wrote this explanation article on how this is supposed to work.

[anatol]

blkinfo for /dev/sr0: type=mbr UUID=44523d4f LABEL= it looks like you still use the old booster. With the new change the filesystem will be detected as iso9660.

Please make sure you pulled issue-223 compiled it, replaced the existing init and then regenerate the image for iso.

[firasuke]

blkinfo for /dev/sr0: type=mbr UUID=44523d4f LABEL= it looks like you still use the old booster. With the new change the filesystem will be detected as iso9660.

Please make sure you pulled issue-223 compiled it, replaced the existing init and then regenerate the image for iso.

Ok I am no longer facing the previous error, instead I am now facing this:

mount(/dev/sr0): read-only file system

[anatol]

It is great to hear about the progress. To handle the latter problem please use ro boot parameter instead of rw. The kernel assumes cdrom is a read-only device and thus requires read-only filesystem.

[firasuke]

It is great to hear about the progress. To handle the latter problem please use ro boot parameter instead of rw. The kernel assumes cdrom is a read-only device and thus requires read-only filesystem.

Ok this is awesome! It is attempting to search for init which is /sbin/init. My custom distribution does not have /sbin and the init is located over at /usr/bin/init.

So I specified init=/usr/bin/init and it worked!

[firasuke]

I guess the service manager at this point should handle the rw remount, as the current filesystem is read-only.

[firasuke]

Also suppose the case the root filesystem was compressed using squashfs or EROFS, would it be possible to include support for this as well? Or perhaps just simply modprobeing the modules erofs and squashfs would suffice?

[anatol]

I am not much familiar with iso handling. But if you provide instructions on how such erofs/squashfs is provisioned, then I can add an integration test to make sure booster handles it as well.

[firasuke]

Thanks for your time and effort!

I will do some more testing, and open a new issue if I encounter another edge case

[anatol]

Sounds good!

And if you provide me instructions how to create an erofs/squashfs ISO file I'll add it to the booster's integration tests.

[firasuke]

Ok, consider the case the root filesystem was compressed using squashfs or erofs into a file called rootfs.erofs.

According to this, I think booster should check whether the root filesystem is compressed/squashed or not, if it is then an additional mount command should be run:

...
mount -t erofs -o ro /dev/some_loop_device /some_mount_directory
...

So in addition to checking for /dev/sr0 and mounting it, booster should also check whether another parameter is passed that specifies the location of the root filesystem on /dev/sr0 whether it was compressed or not (for example another root= as an argument for booster and not for cmdline)..