Use calculated, flexible section-vma offsets instead of hardcoded ones

[conrad-heimbold]

objcopy in https://github.com/andreyv/sbupdate/blob/master/sbupdate#L170 uses hard-coded section-vma positions:

objcopy \
    --add-section .osrel="/etc/os-release"                          --change-section-vma .osrel=0x20000    \
    --add-section .cmdline=<(echo -n "${cmdline}")                  --change-section-vma .cmdline=0x30000  \
    --add-section .splash="${SPLASH}"                               --change-section-vma .splash=0x40000   \
    --add-section .linux="${linux}"                                 --change-section-vma .linux=0x2000000  \
    --add-section .initrd=<(cat "${INITRD_PREPEND[@]}" "${initrd}") --change-section-vma .initrd=0x3000000 \
    "${EFISTUB}" "${output}"

0x20000 for osrel, 0x30000 for cmdline, 0x40000 for splash, and so on.

The Arch Wiki in https://wiki.archlinux.org/title/Unified_kernel_image#Manually however recommends more flexible section-vma positions, that depend on the size of the sections / parts before. They first calculate the offsets:

stub_line=$(objdump -h "/usr/lib/systemd/boot/efi/linuxx64.efi.stub" | tail -2 | head -1)
stub_size=0x$(echo "$stub_line" | awk '{print $3}')
stub_offs=0x$(echo "$stub_line" | awk '{print $4}')
osrel_offs=$((stub_size + stub_offs))
cmdline_offs=$((osrel_offs + $(stat -c%s "/usr/lib/os-release")))
splash_offs=$((cmdline_offs + $(stat -c%s "/etc/kernel/cmdline")))
linux_offs=$((splash_offs + $(stat -c%s "/usr/share/systemd/bootctl/splash-arch.bmp")))
initrd_offs=$((linux_offs + $(stat -c%s "vmlinuz-file")))

... and then use these offsets instead:

objcopy \
    --add-section .osrel="/usr/lib/os-release" --change-section-vma .osrel=$(printf 0x%x $osrel_offs) \
    --add-section .cmdline="/etc/kernel/cmdline" \
    --change-section-vma .cmdline=$(printf 0x%x $cmdline_offs) \
    --add-section .splash="/usr/share/systemd/bootctl/splash-arch.bmp" \
    --change-section-vma .splash=$(printf 0x%x $splash_offs) \
    --add-section .linux="vmlinuz-file" \
    --change-section-vma .linux=$(printf 0x%x $linux_offs) \
    --add-section .initrd="initrd-file" \
    --change-section-vma .initrd=$(printf 0x%x $initrd_offs) \
    "/usr/lib/systemd/boot/efi/linuxx64.efi.stub" "linux.efi"

I think this flexible approach is much better, because it can make the resulting UEFI kernel image much smaller; which makes booting a little bit faster.

Analysing the generated UEFI image and getting the sections is still easily possible with tools like objdump, so I don't see any added value in using hard-coded offsets:

$ objdump -h linux-signed.efi

linux-signed.efi:     file format pei-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00006dd0  0000000000003000  0000000000003000  00000400  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .reloc        0000000c  000000000000a000  000000000000a000  00007200  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .data         000028f0  000000000000b000  000000000000b000  00007400  2**4
                  CONTENTS, ALLOC, LOAD, DATA
  3 .dynamic      00000100  000000000000e000  000000000000e000  00009e00  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  4 .rela         00000ed0  000000000000f000  000000000000f000  0000a000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .dynsym       00000018  0000000000010000  0000000000010000  0000b000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  6 .sbat         00000108  0000000000012000  0000000000012000  0000b200  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  7 .sdmagic      00000030  0000000000012120  0000000000012120  0000b400  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .osrel        0000011d  0000000000020000  0000000000020000  0000b600  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  9 .cmdline      00000070  0000000000030000  0000000000030000  0000b800  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .splash       0005c572  0000000000040000  0000000000040000  0000ba00  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 11 .linux        00a7e340  0000000002000000  0000000002000000  00068000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 12 .initrd       01c6a282  0000000003000000  0000000003000000  00ae6400  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA

Should I make a pull request for this improvement?

[conrad-heimbold]

I just created an EFI kernel image both with hard-coded/fixed offsets and with flexible/calculated offsets and the difference is quite noticeable:

$ ls -hsl /boot/efi/EFI/Arch/
total 153M
35M -rwxr-xr-x 1 root root 35M Aug 22 17:15 linux-calculated-offsets-signed.efi
40M -rwxr-xr-x 1 root root 40M Aug 22 17:14 linux-hardcoded-offsets-signed.efi

So with calculated offsets, the Linux EFI image is approximately 5 Mebibyte smaller. This means, the Linux EFI image is 5 / 40 = 12,5% smaller!

Now I will try to boot both versions to check how much faster the EFI kernel image with flexible/calculated offsets is in comparison to the EFI kernel image with hardcoded/fixed offsets.

[conrad-heimbold]

Boot time with fixed/hardcoded offsets:

Startup finished in 6.391s (firmware) + 449ms (loader) + 647ms (kernel) + 8.396s (initrd) + 1.328s (userspace) = 17.214s 
graphical.target reached after 1.325s in userspace.

Startup finished in 6.336s (firmware) + 449ms (loader) + 630ms (kernel) + 8.759s (initrd) + 1.967s (userspace) = 18.142s 
graphical.target reached after 1.967s in userspace.

Boot time with calculated/flexible offsets:

Startup finished in 6.237s (firmware) + 386ms (loader) + 634ms (kernel) + 7.500s (initrd) + 1.342s (userspace) = 16.101s 
graphical.target reached after 1.315s in userspace.

Startup finished in 6.186s (firmware) + 385ms (loader) + 646ms (kernel) + 8.304s (initrd) + 1.248s (userspace) = 16.770s 
graphical.target reached after 1.247s in userspace.

So approx. 449ms - 386 ms = 63 milliseconds less for the loader, and (8.396s + 8.759s) / 2 - (7.500s + 8.304s) / 2 = 8.5775s - 7.902s = 0.676 seconds less for the initrd.

So using flexible/calculated offsets would make booting the unified Linux EFI image approximately 0.739 seconds faster!

[conrad-heimbold]

For a comparison, the offsets:

$ objdump -h /efi/EFI/Arch/linux-hardcoded-offsets-signed.efi > /efi/EFI/Arch/linux-hardcoded-offsets-signed.efi.offsets 
$ objdump -h /efi/EFI/Arch/linux-calculated-offsets-signed.efi > /efi/EFI/Arch/linux-calculated-offsets-signed.efi.offsets 
$ diff linux-calculated-offsets-signed.efi.offsets linux-hardcoded-offsets-signed.efi.offsets 
2c2
< linux-calculated-offsets-signed.efi:     file format pei-x86-64
---
> linux-hardcoded-offsets-signed.efi:     file format pei-x86-64
22c22
<   8 .osrel        0000011d  0000000000012150  0000000000012150  0000b600  2**2
---
>   8 .osrel        0000011d  0000000000020000  0000000000020000  0000b600  2**2
24c24
<   9 .cmdline      00000071  000000000001226d  000000000001226d  0000b800  2**2
---
>   9 .cmdline      00000070  0000000000030000  0000000000030000  0000b800  2**2
26c26
<  10 .splash       0005c572  00000000000122de  00000000000122de  0000ba00  2**2
---
>  10 .splash       0005c572  0000000000040000  0000000000040000  0000ba00  2**2
28c28
<  11 .linux        00a7e340  000000000006e850  000000000006e850  00068000  2**2
---
>  11 .linux        00a7e340  0000000002000000  0000000002000000  00068000  2**2
30c30
<  12 .initrd       01779682  0000000000aecb90  0000000000aecb90  00ae6400  2**2
---
>  12 .initrd       01c6a282  0000000003000000  0000000003000000  00ae6400  2**2

[conrad-heimbold]

Oh, i just noticed that this won't make the image smaller, because the vma section positions are concerning the image when loaded into RAM, not the file itself. My assumption was wrong. I did it again with the same initramfs-file and now it doesn't make a difference:

$ ls -alh /efi/EFI/Arch/
total 158M
drwxr-xr-x 3 root root 4.0K Aug 22 18:36 .
drwxr-xr-x 9 root root 4.0K Aug 14 21:20 ..
-rwxr-xr-x 1 root root  40M Aug 22 18:35 linux-calculated-offsets-signed.efi
-rwxr-xr-x 1 root root  40M Aug 22 18:34 linux-hardcoded-offsets-signed.efi

The files have the same size, one is not 5 MB / 12,5% smaller.

$ diff boottime-with-hardcoded-efi-offsets.3.txt boottime-with-calculated-efi-offsets.3.txt 
1,2c1,2
< Startup finished in 6.383s (firmware) + 449ms (loader) + 643ms (kernel) + 7.959s (initrd) + 1.078s (userspace) = 16.514s 
< graphical.target reached after 1.076s in userspace.
---
> Startup finished in 6.339s (firmware) + 450ms (loader) + 647ms (kernel) + 8.310s (initrd) + 1.240s (userspace) = 16.987s 
> graphical.target reached after 1.232s in userspace.

I don't know what went wrong. Now the difference is not that big anymore.

[andreyv]

Thanks for the research. I'll look into this further to see if there are any benefits.

[piernov]

After some systemd 254 changes it seems, the signed image produced by sbupdate is broken, issue is similar to dracutdevs/dracut#2431 so it seems like this will be a requirement after systemd 254 release. (254rc3 currently in ArchLinux testing and this results in a non-bootable kernel)

[andreyv]

Unfortunately, I don't have the capability to check it now. If someone can provide a succinct fix, this would be nice.

[Maryse47]

Arch updated to systemd 254 now. If this project won't be adjusted soon then I think it should be retired or big bold warning should be added in readme about its incompatibility with Arch. Thx for all those years.

[drujd]

Just became a victim of this, with a really, REALLY bad timing.

This should either be fixed ASAP or at least deprecated / updated with a script that prints a big warning about not being able to boot...

I think that with systemd 254 rolling to Arch today, this is going to burn a lot of people.

[andreyv]

The tool backs up the previous signed image automatically to a .bak file next to the original image on the EFI partition. You can boot it manually to rescue the system.

[andreyv]

Please try the latest version from AUR.

[jm355]

I get:

/e/E/arch [1]$ sudo sbupdate
Generating and signing linux-signed.efi
Host arch 'x86_64', EFI arch 'x64'
Traceback (most recent call last):
  File "/usr/lib/systemd/ukify", line 1482, in <module>
    main()
  File "/usr/lib/systemd/ukify", line 1472, in main
    check_inputs(opts)
  File "/usr/lib/systemd/ukify", line 359, in check_inputs
    check_splash(opts.splash)
  File "/usr/lib/systemd/ukify", line 342, in check_splash
    img = Image.open(filename, formats=['BMP'])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/PIL/Image.py", line 3280, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/dev/null'

[jm355]

Also if anyone is having trouble using the backup image, once it partially boots and puts you into the root shell you may have to downgrade linux and reboot

pacman -U /var/cache/pacman/pkg/linux-6.4.7.arch1-1-x86_64.pkg.tar.zst

That said, at this point if you set up a uki with mkinitcpio, it's easier to use sbsigntools and boot the uki direcly https://wiki.archlinux.org/title/Unified_kernel_image#Signing_the_UKIs_for_Secure_Boot not sure about keeping fwupdmgr signed though, if anyone has a simple solution for that ~lmk~

[andreyv]

Splash should be fixed in 1bd9722cb4017c545c5ded7eb7c6b66921625dd0.

[bitwave]

Have the same problem with 0.r131.4926075-1, kernel is not bootable. Please fix this ASAP.

[andreyv]

@bitwave What is the console output when you run sudo sbupdate manually?

[andreyv]

@bitwave The fixed version is r132 or later... Please update your installation.