Make it possible to boot from loop-mounted ISO using the FreeBSD boot loader

[probonopd]

It would be very useful (not just for helloSystem but in general) if one could boot into ISO files from the FreeBSD boot loader.

https://reviews.freebsd.org/D19733 seems to implement at least the first half of it:

By mapping file, we create vdiskX device, the device will be listed by lsdev [-v] and can be accessed directly as ls vdisk0p1:/path or can be used as value for currdev variable.

map-vdisk can be used to loop-mount an ISO and to boot the kernel from it. I tested it and it works great:

map-vdisk aaa.iso # vdisk0 gets created
lsdev -v # shows vdisk0
set currdev=vdisk0:
unset vfs.root.mountfrom
boot -v

starts the kernel that is stored inside the ISO. This is the first half of the needed functionality.

But then the kernel does not see the vdisk when trying to mount rootfs... is this functionality not implemented yet?

For this, we need the second half of the functionality.

Could it be ported from the ventoy kernel module?

This is what Ventoy seems to be doing in GRUB; can we achieve the same effect using the FreeBSD bootloader? If yes, then the existing Ventoy FreeBSD kernel module could possibly do the rest of the work?

    vtoy_ssprintf(buf, pos, "hint.ventoy.0.disksize=%llu\n", (ulonglong)(disk->total_sectors * (1 << disk->log_sector_size)));
    vtoy_ssprintf(buf, pos, "hint.ventoy.0.diskuuid=\"%s\"\n", uuid);
    vtoy_ssprintf(buf, pos, "hint.ventoy.0.disksignature=%02x%02x%02x%02x\n", disk_sig[0], disk_sig[1], disk_sig[2], disk_sig[3]);
    vtoy_ssprintf(buf, pos, "hint.ventoy.0.segnum=%u\n", g_img_chunk_list.cur_chunk);

    for (i = 0; i < g_img_chunk_list.cur_chunk; i++)
    {
        chunk = g_img_chunk_list.chunk + i;
        vtoy_ssprintf(buf, pos, "hint.ventoy.%u.seg=\"0x%llx@0x%llx\"\n", 
            i, (ulonglong)(chunk->disk_start_sector * 512),
            (ulonglong)((chunk->disk_end_sector + 1) * 512));
    }

https://github.com/ventoy/Ventoy/blob/a4200ed99eaae5ec2b3034077d9a5ff461d5e8e8/GRUB2/MOD_SRC/grub-2.04/grub-core/ventoy/ventoy_unix.c#L313

[emaste]

Paging @tsoome

[emaste]

Also @trasz.

I suspect this isn't really feasible with reroot - as I understand it, essentially we need to:

• mount a filesystem on the physical device (probably FAT or ISO9660)
• create a vnode md from a file in that filesystem
• use that md as the root mount

[probonopd]

Let's look into how Ventoy is doing it. I think they export certain information in GRUB to well-known memory locations, load the kernel, and then let the Ventoy FreeBSD kernel module pick up the information previously exported. From there on, the Ventoy kernel module does the rest of the work. So it is simply(?) a matter of exporting the same data that Ventoy exports using GRUB. Or so I imagine is how it works, looking at the documentation.

For Legacy BIOS system, Ventoy saves the information in the low-memory, between 0x80000~0xA0000 (phy memory address). For UEFI, Ventoy saves the information to an EFI Variable with NAME and GUID as follows: VentoyOsParam { 0x77772020, 0x2e77, 0x6576, { 0x6e, 0x74, 0x6f, 0x79, 0x2e, 0x6e, 0x65, 0x74 }}

So I believe that if we can get the FreeBSD bootloader to write the information listed in this table to the memory locations described above, then it should "just work". (helloSystem already ships the Ventoy FreeBSD kernel module.)

[emaste]

Huh, from g_ventoy.h:

#define VENTOY_UNIX_MAX_SEGNUM   40960
struct g_ventoy_seg {
    uint64_t seg_start_bytes;
    uint64_t seg_end_bytes;
};

struct g_ventoy_map{
    uint32_t magic1[4];
    uint32_t magic2[4];
    uint64_t segnum;
    uint64_t disksize;
    uint8_t diskuuid[16];
    struct g_ventoy_seg seglist[VENTOY_UNIX_MAX_SEGNUM];
    uint32_t magic3[4];

so it looks like the Ventoy geom makes use of a map of (up to 40960) byte ranges to translate between image file offsets and offsets in the underlying device -- clever, but it means that some Ventoy component needs to generate those maps (either after the image file is copied to the device, or as part of the boot process prior to FreeBSD starting). AFAICT this won't work with only the loader's vdisk support, without involvement of Ventoy.

[probonopd]

That's what I mean - some code would probably have to be added to the bootloader. Not sure whether it could be done by Lua scripting alone.

[tsoome]

We do not need UEFI vars or "well known" memory areas - we do have loader environment passed to kernel and we can use that mechanism just fine. The variables can be populated/removed with map/unmap commands, and all the kernel module needs to do, is to pick up the values (just like many other kernel modules are already doing). grub has no mechanism to pass data to loaded kernel, thats why they need alternate mechanisms.

The problem with memory area is, we can not be sure it is actually free and usable. And the problem with UEFI variables is, the variable storage can get fragmented (we need to add, then remove related vars and data) and this can lead to issues with system firmware.

[probonopd]

Great. So one would need to:

• Write code to compute and export the needed loader variables (can this be done in lua or is this only/better done in C?)
• Augment the Ventoy kernel module to (preferably) read and act upon those loader variables

Correct?

[emaste]

I don't object to Ventoy, but would like to see if there's a way we can handle the kernel's side of map-vdisk without relying on Ventoy. (@tsoome I'm curious if you can explain how Illumos made use of this.)

(I'm still unsure if Ventoy generates the offset mapping info on each boot, or it's done as some sort of post-processing step after copying the ISO to the USB stick.)

[tsoome]

Great. So one would need to:

• Write code to compute and export the needed loader variables (can this be done in lua or is this only/better done in C?)

That depends on what exactly we would need from kernel [module] point of view. By naive approach, I'd guess, the file name could be just enough and if so, it could be done by script which does create mapping.

• Augment the Ventoy kernel module to (preferably) read and act upon those loader variables

Correct?

Yes.

[tsoome]

I don't object to Ventoy, but would like to see if there's a way we can handle the kernel's side of map-vdisk without relying on Ventoy. (@tsoome I'm curious if you can explain how Illumos made use of this.)

(I'm still unsure if Ventoy generates the offset mapping info on each boot, or it's done as some sort of post-processing step after copying the ISO to the USB stick.)

I am not aware if any illumos distribution is actually using it. Someone was complaining that there is no way to use mapped file, so I did add the feature and did test it is possible to boot from it. However, there is one important difference compared to FreeBSD - in case of illumos, we load kernel and boot archive (as ramdisk), and in case of iso, the boot archive is actually miniroot with set of tools - so all we need to do really in illumos, is to pass mapped file name and device ID, and script the file mapping to disk in userland...

[emaste]

in case of illumos, we load kernel and boot archive (as ramdisk)

Ah, ok - it is possible (but atypical) to do the same in FreeBSD. I just wanted to make sure I wasn't missing something.

[probonopd]

I'm still unsure if Ventoy generates the offset mapping info on each boot, or it's done as some sort of post-processing step after copying the ISO to the USB stick.

Presumably a GRUB (which Ventoy is based upon) plugin.

There is no extra step (like running a script) for the user - just copy an ISO onto the Ventoy volume, reboot.

ramdisk

Ventoy does not require one to copy the whole ISO to ram. Especially for larger ISOs that makes a large difference.