search

weldr / lorax

Tools for creating images, including the Anaconda boot.iso, live disk images, iso's, and filesystem images.
GNU General Public License v2.0
295 stars 158 forks source link

The live ISO images generated by livemedia-creator are unbootable if the squashed root filesystem is over 4GB #1353

Open jkool702 opened 12 months ago

jkool702 commented 12 months ago

Problem: when generating a custom live ISO image, using the standard "squashfs root filesystem" method, if the squashed root filesystem is over 4 GB in size the live ISO will not be bootable.

Cause: having a file larger than 4 GB necessitates having /boot be on a NTFS filesystem instead of a FAT32 filesystem, and the GRUB binary added to the ESP does not have NTFS support. If a grub NTFS module is being added, then it is getting added to the NTFS /boot partition where it doesnt do any good. Unfortunately, the kernel/initramfs (which has ntfs support compiled in) is also on the NTFS /boot partition. Ultimately, this means that when you try to boot an image that (by necessity) has a NTFS /boot partition you get dropped into a GRUB recovery shell where you cant do anything because there is nothing on the ESP that GRUB can utalize to read that NTFS partition.

Possible Solutions: It seems like there are 2-3 fairly obvious possible solutions to this:

  1. Make it so that GRUB can read data on a NTFS filesystem (in a way that doesnt require "accessing something on a NTFS filesystem" as a prerequisite). I'd imagine this involves either a) adding the GRUB NTFS module to the ESP and adding insmod ntfs to the GRUB menu) or b) compiling GRUB NTFS support into the grub EFI image on the ESP.
  2. Move the kernel and initramfs onto a FAT32 partition (the ESP seems like it would be the obvious choice, though adding another /boot-esque partition for it could work too), while keeping the squashfs root filesystem image on the NTFS /boot partition. This would also require ensuring that the kernel has ntfs support compiled in and perhaps modifying/adding a dracut routine that tells the kernel to mount the NTFS /boot partition to find the squashfs root filesystem image.
  3. Im not sure if this would work, but perhaps replacing the grub efi (e.g., at /boot/EFI/BOOT/BOOTX64.EFI) with a unified kernel image (that has the same name and has NTFS support compiled in and knows it needs to mount the NTFS /boot partition to find the squashfs root filesystem image).
bcl commented 12 months ago

Can you describe what you are doing in a bit more detail and maybe include relevant logs from trying to boot? In the live iso the UEFI binaries are on the iso under /EFI, and in the embedded efiboot.img so booting them shouldn't have anything to do with the size of the squashfs.img . The squashfs image is then mounted by the initramfs and at that point there should be no FAT32 involved. If the squashfs.img is > 4G the iso should be building with -iso-level 3 in order to support larger files, maybe what you are seeing is related to that?

jkool702 commented 12 months ago

So, I looked into this a bit more. Its probably not actually the fault of livemedia-creator, though it is much easier to fix from the livemedia-creator side.

If you dd the image onto a drive or boot it with QEMU/KVM it works. But, I was using Rufus in "ISO image" mode to write the ISO to disk. This mode doesnt just dd the ISO image, but rather creates filesystem(s) on the USB and then copies on the files. This makes it so the "extra" space on the USB beyond what the ISO image needs is still usable (no "shrinking the partition down to the ISO image size) and is writeable (the root filesystem isnt, since that is still on a squashfs, but the /boot partition where the squashfs.img file is is writeable). Which is really nice.

Rufus tries to mimic a standard linux EFI setup here, and as such puts /EFI (which contains the bootable efi images) on a FAT32 ESP partition, and everything else on a partition resembling /boot. Rufus is on windows, so for the /boot partition its only choices are FAT32 or NTFS, and if you have any files above 4gb in size that choice drops to just NTFS.

Unfortunately, the GRUB NTFS module (and all the other GRUB modules) at /boot/grub2/<arch> as well as the kernel/initramfs at /images/pxeboot are on that NTFS /boot partition, and as such GRUB literally cant do anything and the drive is unbootable.

I think the easiest solution would be to on EFI systems include a copy of /boot/grub2 under /EFI/BOOT (and tell GRUB to look for them there), so that the grub modules get added to the ESP, which allows GRUB to read the NTFS partition that has the kernel/initramfs/squashfs.img. Itll add ~3 MB to the ISO image, but that seems not too bad.