limine-bootloader / limine

Modern, advanced, portable, multiprotocol bootloader and boot manager.
https://limine-bootloader.org
BSD 2-Clause "Simplified" License
1.85k stars 141 forks source link

Potentially broken EFI memory map #356

Closed phip1611 closed 3 months ago

phip1611 commented 5 months ago

Hi, I'm currently not sure where the problem is and maybe you have something helpful to say. I think Limine might have a bug when it comes to the EFI memory map and Multiboot2.

VM configuration

I'm booting a QEMU x86_64 VM with the q35 machine type, 128MB of RAM, and OVMF as UEFI-firmware in version 202402. The firmware boots Limine 7.5.2 from a hybrid ISO image. (I think that Limine boots the legacy way as OVMF has the Compatibility Support Module (CSM) active, hence, the .EFI file is not touched.)

Limine Payload

Limine than boots a Multiboot2 binary which is mapped straight into memory without any relocation at 16MB. The handoff happens in i386 32-bit protected machine state without paging.

In that payload, the EFI memory map coming from the Multiboot Boot Information (MBI) is corrupt. The relevant output, containing the regular memory map (which seems fine) and the EFI memory map, is the following:

The tags from the MBI:

Note that the efi_memory_map seems to be suspiously big.

[DEBUG: multiboot2_payload/src/main.rs@32]: multiboot2_hdr=0x00010000, multiboot2_magic=0x36d76289
Multiboot2BootInformation {
    start_address: 0x10000,
    end_address: 0x11a70,
    total_size: 0x1a70,
    // ...
    efi_memory_map: Some(
        EFIMemoryMapTag {
            typ: EfiMmap,
            size: 0x1540,
            desc_size: 0x30,
            desc_version: 0x1,
            memory_map: "<data...>",
        },
    ),
    // ...
    memory_map: Some(
        MemoryMapTag {
            typ: Mmap,
            size: 0x268,
            entry_size: 0x18,
            entry_version: 0x0,
            areas: [
                // ...
            ],
        },
    ),
}

Printed legacy memory map (seems fine):

``` [ INFO: multiboot2_payload/src/verify/mod.rs@20]: loaded by Limine Memory Map (legacy): 0x0000000000 - 0x00000a0000 ( 0.6 MiB Available) 0x0000100000 - 0x0000800000 ( 7.0 MiB Available) 0x0000800000 - 0x0000808000 ( 0.0 MiB ReservedHibernate) 0x0000808000 - 0x000080b000 ( 0.0 MiB Available) 0x000080b000 - 0x000080c000 ( 0.0 MiB ReservedHibernate) 0x000080c000 - 0x0000810000 ( 0.0 MiB Available) 0x0000810000 - 0x0000900000 ( 0.9 MiB ReservedHibernate) 0x0000900000 - 0x0006448000 ( 91.3 MiB Available) 0x0006448000 - 0x000644d000 ( 0.0 MiB Reserved) 0x000644d000 - 0x0006456000 ( 0.0 MiB Available) 0x0006456000 - 0x000648e000 ( 0.2 MiB Reserved) 0x000648e000 - 0x00064bc000 ( 0.2 MiB Available) 0x00064bc000 - 0x00064c7000 ( 0.0 MiB Reserved) 0x00064c7000 - 0x00074ed000 ( 16.1 MiB Available) 0x00074ed000 - 0x00075ed000 ( 1.0 MiB Reserved) 0x00075ed000 - 0x00076ed000 ( 1.0 MiB Reserved) 0x00076ed000 - 0x000776d000 ( 0.5 MiB Reserved) 0x000776d000 - 0x000777f000 ( 0.1 MiB AcpiAvailable) 0x000777f000 - 0x00077ff000 ( 0.5 MiB ReservedHibernate) 0x00077ff000 - 0x0007ef4000 ( 7.0 MiB Available) 0x0007ef4000 - 0x0007f78000 ( 0.5 MiB Reserved) 0x0007f78000 - 0x0008000000 ( 0.5 MiB ReservedHibernate) 0x00e0000000 - 0x00f0000000 (256.0 MiB Reserved) 0x00feffc000 - 0x00ff000000 ( 0.0 MiB Reserved) 0xfd00000000 - 0x10000000000 (12288.0 MiB Reserved) ```

Printed EFI memory map (seems odd):

``` Memory Map (EFI): 0x0000000000000000 - 0x0000100000000000 (16777216.0 MiB BOOT_SERVICES_CODE) 0x0000100000000000 - 0x000a000000000000 (2667577344.0 MiB CONVENTIONAL) 0x0010000000000000 - 0x0080000000000000 (30064771072.0 MiB CONVENTIONAL) 0x0080000000000000 - 0x0080800000000000 (134217728.0 MiB ACPI_NON_VOLATILE) 0x0080800000000000 - 0x0080b00000000000 (50331648.0 MiB CONVENTIONAL) 0x0080b00000000000 - 0x0080c00000000000 (16777216.0 MiB ACPI_NON_VOLATILE) 0x0080c00000000000 - 0x0081000000000000 (67108864.0 MiB CONVENTIONAL) 0x0081000000000000 - 0x0090000000000000 (4026531840.0 MiB ACPI_NON_VOLATILE) 0x0090000000000000 - 0x0178000000000000 (62277025792.0 MiB BOOT_SERVICES_DATA) 0x0178000000000000 - 0x0237500000000000 (51355058176.0 MiB CONVENTIONAL) 0x0237500000000000 - 0x03b7500000000000 (103079215104.0 MiB CONVENTIONAL) 0x03b7500000000000 - 0x03b9500000000000 (536870912.0 MiB BOOT_SERVICES_DATA) 0x03b9500000000000 - 0x03c5600000000000 (3238002688.0 MiB CONVENTIONAL) 0x03c5600000000000 - 0x0644800000000000 (171563810816.0 MiB CONVENTIONAL) 0x0644800000000000 - 0x0644d00000000000 (83886080.0 MiB LOADER_DATA) 0x0644d00000000000 - 0x0645600000000000 (150994944.0 MiB BOOT_SERVICES_DATA) 0x0645600000000000 - 0x0648e00000000000 (939524096.0 MiB LOADER_CODE) 0x0648e00000000000 - 0x064c700000000000 (956301312.0 MiB LOADER_DATA) 0x064c700000000000 - 0x068ea00000000000 (17767071744.0 MiB BOOT_SERVICES_DATA) 0x068ea00000000000 - 0x0699200000000000 (2818572288.0 MiB BOOT_SERVICES_CODE) 0x0699200000000000 - 0x06a0600000000000 (1946157056.0 MiB BOOT_SERVICES_DATA) 0x06a0600000000000 - 0x06a1f00000000000 (419430400.0 MiB BOOT_SERVICES_CODE) 0x06a1f00000000000 - 0x06a2400000000000 (83886080.0 MiB BOOT_SERVICES_DATA) 0x06a2400000000000 - 0x06a3400000000000 (268435456.0 MiB BOOT_SERVICES_CODE) 0x06a3400000000000 - 0x06a3800000000000 (67108864.0 MiB BOOT_SERVICES_DATA) 0x06a3800000000000 - 0x06a5100000000000 (419430400.0 MiB BOOT_SERVICES_CODE) 0x06a5100000000000 - 0x06a5200000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x06a5200000000000 - 0x06a5500000000000 (50331648.0 MiB BOOT_SERVICES_CODE) 0x06a5500000000000 - 0x06a5600000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x06a5600000000000 - 0x06a8900000000000 (855638016.0 MiB BOOT_SERVICES_CODE) 0x06a8900000000000 - 0x06a9300000000000 (167772160.0 MiB BOOT_SERVICES_DATA) 0x06a9300000000000 - 0x06a9900000000000 (100663296.0 MiB BOOT_SERVICES_CODE) 0x06a9900000000000 - 0x06a9f00000000000 (100663296.0 MiB BOOT_SERVICES_DATA) 0x06a9f00000000000 - 0x06aaf00000000000 (268435456.0 MiB BOOT_SERVICES_CODE) 0x06aaf00000000000 - 0x06ab200000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06ab200000000000 - 0x06acd00000000000 (452984832.0 MiB BOOT_SERVICES_CODE) 0x06acd00000000000 - 0x06ace00000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x06ace00000000000 - 0x06b0000000000000 (838860800.0 MiB BOOT_SERVICES_CODE) 0x06b0000000000000 - 0x06b0300000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06b0300000000000 - 0x06b0a00000000000 (117440512.0 MiB BOOT_SERVICES_CODE) 0x06b0a00000000000 - 0x06b0d00000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06b0d00000000000 - 0x06b1c00000000000 (251658240.0 MiB BOOT_SERVICES_CODE) 0x06b1c00000000000 - 0x06b1d00000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x06b1d00000000000 - 0x06b1f00000000000 (33554432.0 MiB BOOT_SERVICES_CODE) 0x06b1f00000000000 - 0x06b2100000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06b2100000000000 - 0x06b2700000000000 (100663296.0 MiB BOOT_SERVICES_CODE) 0x06b2700000000000 - 0x06b2a00000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06b2a00000000000 - 0x06b2f00000000000 (83886080.0 MiB BOOT_SERVICES_CODE) 0x06b2f00000000000 - 0x06b3100000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06b3100000000000 - 0x06b3d00000000000 (201326592.0 MiB BOOT_SERVICES_CODE) 0x06b3d00000000000 - 0x06b3f00000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06b3f00000000000 - 0x06b4d00000000000 (234881024.0 MiB BOOT_SERVICES_CODE) 0x06b4d00000000000 - 0x06b4f00000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06b4f00000000000 - 0x06b5100000000000 (33554432.0 MiB BOOT_SERVICES_CODE) 0x06b5100000000000 - 0x06b5300000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06b5300000000000 - 0x06b6f00000000000 (469762048.0 MiB BOOT_SERVICES_CODE) 0x06b6f00000000000 - 0x06b7200000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06b7200000000000 - 0x06b8700000000000 (352321536.0 MiB BOOT_SERVICES_CODE) 0x06b8700000000000 - 0x06b8a00000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06b8a00000000000 - 0x06b9100000000000 (117440512.0 MiB BOOT_SERVICES_CODE) 0x06b9100000000000 - 0x06b9900000000000 (134217728.0 MiB BOOT_SERVICES_DATA) 0x06b9900000000000 - 0x06b9d00000000000 (67108864.0 MiB BOOT_SERVICES_CODE) 0x06b9d00000000000 - 0x06ba300000000000 (100663296.0 MiB BOOT_SERVICES_DATA) 0x06ba300000000000 - 0x06ba700000000000 (67108864.0 MiB BOOT_SERVICES_CODE) 0x06ba700000000000 - 0x06baa00000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06baa00000000000 - 0x06be900000000000 (1056964608.0 MiB BOOT_SERVICES_CODE) 0x06be900000000000 - 0x06bf000000000000 (117440512.0 MiB BOOT_SERVICES_DATA) 0x06bf000000000000 - 0x06bf300000000000 (50331648.0 MiB BOOT_SERVICES_CODE) 0x06bf300000000000 - 0x06bf600000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06bf600000000000 - 0x06bfb00000000000 (83886080.0 MiB BOOT_SERVICES_CODE) 0x06bfb00000000000 - 0x06bfd00000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06bfd00000000000 - 0x06c0000000000000 (50331648.0 MiB BOOT_SERVICES_CODE) 0x06c0000000000000 - 0x06e0300000000000 (8640266240.0 MiB BOOT_SERVICES_DATA) 0x06e0300000000000 - 0x06e1800000000000 (352321536.0 MiB BOOT_SERVICES_CODE) 0x06e1800000000000 - 0x06e1900000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x06e1900000000000 - 0x06e1d00000000000 (67108864.0 MiB BOOT_SERVICES_CODE) 0x06e1d00000000000 - 0x06e1f00000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06e1f00000000000 - 0x06e2700000000000 (134217728.0 MiB BOOT_SERVICES_CODE) 0x06e2700000000000 - 0x06e2a00000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x06e2a00000000000 - 0x06e2b00000000000 (16777216.0 MiB BOOT_SERVICES_CODE) 0x06e2b00000000000 - 0x06e2d00000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06e2d00000000000 - 0x06e3100000000000 (67108864.0 MiB BOOT_SERVICES_CODE) 0x06e3100000000000 - 0x06e3300000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x06e3300000000000 - 0x06e4a00000000000 (385875968.0 MiB BOOT_SERVICES_CODE) 0x06e4a00000000000 - 0x06e4b00000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x06e4b00000000000 - 0x06e4c00000000000 (16777216.0 MiB BOOT_SERVICES_CODE) 0x06e4c00000000000 - 0x0726000000000000 (17515413504.0 MiB BOOT_SERVICES_DATA) 0x0726000000000000 - 0x0726600000000000 (100663296.0 MiB BOOT_SERVICES_CODE) 0x0726600000000000 - 0x0726900000000000 (50331648.0 MiB BOOT_SERVICES_DATA) 0x0726900000000000 - 0x0726a00000000000 (16777216.0 MiB BOOT_SERVICES_CODE) 0x0726a00000000000 - 0x0726b00000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x0726b00000000000 - 0x0727500000000000 (167772160.0 MiB BOOT_SERVICES_CODE) 0x0727500000000000 - 0x0727700000000000 (33554432.0 MiB BOOT_SERVICES_DATA) 0x0727700000000000 - 0x0727900000000000 (33554432.0 MiB BOOT_SERVICES_CODE) 0x0727900000000000 - 0x0727a00000000000 (16777216.0 MiB BOOT_SERVICES_DATA) 0x0727a00000000000 - 0x0727d00000000000 (50331648.0 MiB BOOT_SERVICES_CODE) 0x0727d00000000000 - 0x074ed00000000000 (10468982784.0 MiB BOOT_SERVICES_DATA) 0x074ed00000000000 - 0x075ed00000000000 (4294967296.0 MiB RUNTIME_SERVICES_DATA) 0x075ed00000000000 - 0x076ed00000000000 (4294967296.0 MiB RUNTIME_SERVICES_CODE) 0x076ed00000000000 - 0x0776d00000000000 (2147483648.0 MiB RESERVED) 0x0776d00000000000 - 0x0777f00000000000 (301989888.0 MiB ACPI_RECLAIM) 0x0777f00000000000 - 0x077ff00000000000 (2147483648.0 MiB ACPI_NON_VOLATILE) 0x077ff00000000000 - 0x07e0000000000000 (25786580992.0 MiB BOOT_SERVICES_DATA) 0x07e0000000000000 - 0x07e8700000000000 (2264924160.0 MiB CONVENTIONAL) 0x07e8700000000000 - 0x07ea700000000000 (536870912.0 MiB BOOT_SERVICES_DATA) 0x07ea700000000000 - 0x07eca00000000000 (587202560.0 MiB BOOT_SERVICES_CODE) 0x07eca00000000000 - 0x07edb00000000000 (285212672.0 MiB BOOT_SERVICES_DATA) 0x07edb00000000000 - 0x07ef400000000000 (419430400.0 MiB BOOT_SERVICES_CODE) 0x07ef400000000000 - 0x07f7800000000000 (2214592512.0 MiB RUNTIME_SERVICES_DATA) 0x07f7800000000000 - 0x0800000000000000 (2281701376.0 MiB ACPI_NON_VOLATILE) 0xe000000000000000 - 0xf000000000000000 (1099511627776.0 MiB RESERVED) 0xfeffc00000000000 - 0xff00000000000000 (67108864.0 MiB RESERVED) 0x0000000000000000 - 0x0000000000000000 ( 0.0 MiB RESERVED) ```

I'm confident that it is not a bug of my code. Especially the multiboot2 code is heavily unit- and integration tested. And I don't have any other weird side-effects.

Do you think, it could be that Limine somehow messes with the EFI mmap?

phip1611 commented 5 months ago

I run Limine with VERBOSE=yes, but all it tells me is:

multiboot2: Loading kernel `boot:///kernel`...
acpi: Found SMBIOS 32-bit entry point at 0x753f000
acpi: Found SMBIOS 64-bit entry point at 0x753d000

According to the boot information, the boot services are already excited (the boot services not exited tag is not present), an efi_std64 tag is present, and an efi_ih32 tag (bugfix PR open) is present.

mintsuki commented 5 months ago

Hi, I just tested this, it seems fine to me, booting a test kernel written in C.

phip1611 commented 5 months ago

Okay, thanks!

I'll do more experiments, including a hand-off without exited boot services, more logging in Limine etc, and come back to you. Likely within the next 4 days.

mintsuki commented 5 months ago

Alright, mind you though that Limine does not support hand-off without exited boot services (sort of by design, because it is a useless, rarely used mode of multiboot2).

phip1611 commented 5 months ago

Ah, oh really? GRUB supports it as well. However, GRUB doesn't support some Multiboot-scenarios that Limine supports on the other hand. 🤷🏻

mintsuki commented 5 months ago

GRUB2 is supposed to be the reference implementation... (let's ignore the places where IT IS the spec and the spec is useless)

phip1611 commented 5 months ago

From my testing, it is hard to get output from Limine. Perhaps I'm missing something. I use the serial output and capture everything in a file with QEMU.

My observations:

TL;DR: Getting output is hard. And tips?

--

By using the following patch to print the EFI memory map that Limine sees:

diff --git a/common/protos/multiboot2.c b/common/protos/multiboot2.c
index e5f085f6..2e7d13a6 100644
--- a/common/protos/multiboot2.c
+++ b/common/protos/multiboot2.c
@@ -558,6 +558,49 @@ reloc_fail:
         append_tag(info_idx, module_tag);
     }

+    //////////////////////////////////////////////
+    // Create EFI memory map tag
+    //////////////////////////////////////////////
+#if defined (UEFI)
+    {
+        if ((efi_mmap_size / efi_desc_size) > MEMMAP_MAX) {
+            panic(false, "multiboot2: too many EFI memory map entries");
+        }
+        if (efi_mmap_size % efi_desc_size != 0) {
+            panic(false, "multiboot2: invalid EFI memory map size");
+        }
+
+        // Create the EFI memory map tag.
+        uint32_t size = sizeof(struct multiboot_tag_efi_mmap) + efi_mmap_size;
+        struct multiboot_tag_efi_mmap *mmap_tag = (struct multiboot_tag_efi_mmap *)(mb2_info + info_idx);
+
+        mmap_tag->type = MULTIBOOT_TAG_TYPE_EFI_MMAP;
+        mmap_tag->descr_vers = efi_desc_ver;
+        mmap_tag->descr_size = efi_desc_size;
+        mmap_tag->size = size;
+
+        print("multiboot2: efi_mmap: \n");
+
+        // Copy over the EFI memory map.
+        memcpy(mmap_tag->efi_mmap, efi_mmap, efi_mmap_size);
+        append_tag(info_idx, mmap_tag);
+
+        print("UEFI memory map: \n");
+        print("  <#> <phys> <size> <type> <attr>\n");
+        int n_entries = efi_mmap_size / efi_desc_size;
+        print("number of entries   = %d\n", n_entries);
+        print("tag size            = %x\n", size);
+        print("efi memory map size = %x\n", efi_mmap_size);
+
+        // Limine serial log output is weird. In the serial.txt file, there
+        // are never printed more than
+        for (int i = 0; i < n_entries; i++) {
+            EFI_MEMORY_DESCRIPTOR * e = ((char *) efi_mmap) + i * efi_desc_size;
+            print("  [%d] %x %x %x %x: \n", i, e->PhysicalStart, e->NumberOfPages * 4096, e->Type, e->Attribute);
+        }
+    }
+#endif
+
     //////////////////////////////////////////////
     // Create command line tag
     //////////////////////////////////////////////
@@ -857,29 +900,7 @@ skip_modeset:;
         append_tag(info_idx, tag);
     }

-    //////////////////////////////////////////////
-    // Create EFI memory map tag
-    //////////////////////////////////////////////
-#if defined (UEFI)
-    {
-        if ((efi_mmap_size / efi_desc_size) > MEMMAP_MAX) {
-            panic(false, "multiboot2: too many EFI memory map entries");
-        }

-        // Create the EFI memory map tag.
-        uint32_t size = sizeof(struct multiboot_tag_efi_mmap) + efi_mmap_size;
-        struct multiboot_tag_efi_mmap *mmap_tag = (struct multiboot_tag_efi_mmap *)(mb2_info + info_idx);
-
-        mmap_tag->type = MULTIBOOT_TAG_TYPE_EFI_MMAP;
-        mmap_tag->descr_vers = efi_desc_ver;
-        mmap_tag->descr_size = efi_desc_size;
-        mmap_tag->size = size;
-
-        // Copy over the EFI memory map.
-        memcpy(mmap_tag->efi_mmap, efi_mmap, efi_mmap_size);
-        append_tag(info_idx, mmap_tag);
-    }
-#endif

     //////////////////////////////////////////////
     // Create network info tag

I could verify that at least the first 8 entries (couldn't get more output, see above) are equal to what my Rust binary kernel finds in multiboot_tag_mmap (not multiboot_tag_efi_mmap).

However, multiboot_tag_efi_mmap reports the correct length and desc_size, but it contains just gibberish.

At this point I can say that it is not the multiboot2 crate as it is thoroughly tested.

Observations:

  1. I'm surprised that Limine provides multiboot_tag_mmap by transforming the EFI memory map into the old format
  2. I think Limine somehow messes with the EFI memory.

I wish I could provide you a minimal example. For now, all I got is this:

which you can run using nix-shell --run "integration-test/run.sh"; cat integration-test/serial.txt

In the output (if you scroll a little up) you can see how the corrupted EFI memory map is printed).

mintsuki commented 5 months ago

I added prints here https://github.com/limine-bootloader/limine/blob/c204af454fc9c11b8ef3633664b6e03817c33ff1/test/multiboot2.c#L102 and it outputs perfectly fine.

mintsuki commented 5 months ago

Just make the test image and run the multiboot2 test from the boot menu, you'll get console output via port 0xe9.

phip1611 commented 5 months ago

Ah, the debugcon device. Nice, thanks! Didn't know the function e9_printf() exists.

phip1611 commented 5 months ago

PS: I was talking about output from the bootloader, not the multiboot2 test binary! I had to move e9print.h and e9print.c to common so that I can use it from common/protos/multiboot2.c - it works. I hope I can find any insights

mintsuki commented 5 months ago

In that case you could've built Limine with E9_OUTPUT=true passed to make (do a make clean first).

phip1611 commented 5 months ago

Please have a look at https://github.com/limine-bootloader/limine/pull/358 - the provided log clearly shows that there is a bug

mintsuki commented 5 months ago

Yes but that's an issue with the printf function we use for testing (which barely works for 32-bit code which this is, since it was made originally as a quick and dirty 64-bit code printf), other than that, the actual values of the memory map look good to me.

The reason they don't look good to you is that the code you posted prints them all at once instead of on several separate printf calls unlike my code.

mintsuki commented 3 months ago

I'll close this as it is most likely an invalid issue.