RFC: support DTB loading

[lumag]

WoA (Windows on ARM) platforms use UEFI+ACPI for booting Windows, however on Qualcomm-bssed platforms the ACPI tables were found to be not suitable for Linux because of the heave PEP usage. For those platforms we provide DTS files in the Linux kernel tree. The bootloader chain needs to load corresponding DTB file before transferring control to Linux. On Lenovo X13s the DTB can be loaded by the UEFI, other laptops and devkits require additional steps in the bootloader chain.

One of the obvious ways to do it is GRUB (or any other bootloader). However in the past GRUB developers pointed out that this doesn't play well with the SecureBoot.

In the past @robclark has developed special application, DtbLoader, later it was reworked by @TravMurav. This app uses DMI ids to select and load the DTB. However this doesn't play well with the distributions and SecureBoot. Distro's shim is set up to load the grub binary, so additional dirty tricks are required.

Does it make sense to integrate a part of such functionality into the shim loader itself? This way we can load the DTB (if required and if it exists) in a standard way, verify the signature against MOK or distro key and then pass control to the next bootloader in a standard way, saving us from all the troubles.

[TravMurav]

With my dtbloader dev hat on:

Does it make sense to integrate a part of such functionality into the shim loader itself?

Quickly reading the readme for the shim, I see that the sole goal of the shim is to transfer chain of trust from the system certificate (i.e. microsoft whatever) to built-in certificate (i.e. redhat), so I'd assume that detecting devices and loading dtb would be out-of-scope for the shim. However I do think that shim could provide us some help:

I'd argue that an interesting proposal would be to add driver loading to the shim instead:

• Shim loads a set of drivers (either a hardcoded list of paths or all in some directory on esp), validating each using the same mechanism as for the next stage efi
• Distro provides (i.e.) dtbloader.efi signed correctly with their key + a /dtbs/ dir
• dtbloader.efi upon loading detects the device and loads the correct dtb, validates that this dtb's hash matches.
• shim boots the next bootloader and magic happens as usual with the correct dtb already in the uefi config table.

This would be simpler on the shim side (i.e. I assume it's not trivial to re-validate and re-sign things with MS certificate every time shim updates) and allow one to add more different drivers later if need be.

Then, assuming, dtbloader is used, the only question left would be to figure out a way to extend the chain of trust to dtb files. Currently dtbloader security model is as follows:

• load the dtb into ram and hash it
• compare the hash with one stored in a efivar
• if hash doesn't match, pause boot and prompt the user to continue with new dtb
• save new hash in efivar

This has a benefit of being absolutely simple but also obvious drawbacks:

• user has to confirm dtb at least the first time image is booted,
• unless software updates the efivar correctly after updating the dtb, user needs to confirm all updates too.

I've briefly considered some alternative solutions too, like having some signed hash file and validating that, then validating the dtb against the hash in the file (i.e. load a data-only .efi and validate it's signature using uefi/shim facilities) but there was no point implementing a complicated solution in dtbloader yet since there is no demand for it as of today.

As a side note, I believe we may not even need to touch shim at all: For example systemd-boot allows loading arbitrary efi drivers before boot, and, I believe, will even helpfully add them to the db if they aren't correctly signed, so if distro uses sd-boot, making use of dtbloader is trivial. (We currently experiment with that in postrmarketOS, however we don't implement secureboot yet). I don't know if grub provides similar mechanism but it could be a reasonable addition there too.

In any case I'm happy to participate in the discussion and see whether we can implement something in dtbloader or in some other way.

[lumag]

I think that if SB is enabled, then the DTB should also be signed (e.g. by using the external PKCS7 signature).

Other than that, let's wait for shim maintainers response.

[AdrianVovk]

CC @calebccff

[calebccff]

One downside to this approach is that U-Boot doesn't support EFI drivers afaik. It just panics when trying to load them via systemd-boot.

I discussed a few different approaches with Adrian before and it seems like having dtbs be signed by the distro and loaded by shim would make a lot of sense. DMI matching will always be best effort though

[TravMurav]

It just panics when trying to load them via systemd-boot.

I believe this works correctly for me, the pmOS trailblazer image with dtbloader included correctly loads dtbloader, which correctly bails out since the device can't be matched:

=> load mmc 1 $loadaddr efi/boot/bootaa64.efi
83456 bytes read in 24 ms (3.3 MiB/s)
=> bootefi $loadaddr                         
Failed to populate board hwids: Not Found
Failed to detect this device!
EFI stub: Decompressing Linux Kernel...
EFI stub: EFI_RNG_PROTOCOL unavailable
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services...
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd030]
[    0.000000] Linux version 6.11.0-rc2-next-20240809 (pmos@prometheus) (aarch64-alpine-linux-musl-gcc (Alpine 14.2.0) 14.2.0, GNU ld (GNU Binutils) 2.42) #2-postmarketOS SMP PREEMPT Wed Aug 14 14:24:15 UTC 2024
[    0.000000] KASLR disabled due to lack of seed
[    0.000000] Machine model: Longcheer L8150

However in this case I'd argue that u-boot could as well load the correct dtb in that case, leaving dtbloader et al out of the picture.

@calebccff note that if the only driver you attempted to load was older versions of (my) dtbloader, it unfortunately /did/ have a bug that would've caused uefi to crash on some devices... I'm sorry if that's what caused this confusion.

[lumag]

I think loading drivers from shim might enlarge possible attack surface.

Devicetree loading should be restricted if SB is enabled. So unless DtbLoader implements proper verification, it's highly likely that major distros will refuse to sign with the distro key the DtbLoader.efi which doesn't verify DTB (see Ubuntu, Debian ), and if it's unsigned, we are back to starting point.

Moreover, existing SHIM_LOCK protocol works only with the PE binaries. In order to verify signatures on DTB files we will need to extend it to v2, adding DTB and/or PKCS7 support. Providing such API might be desired or it might not. Having DtbLoader inside the shim removes the need for such API extensions.

[AdrianVovk]

Another part of the difficulty is that DTB isn't necessary distro agnostic. Sure, in theory it's supposed to be, but in practice it's kinda tied to the kernel version

The idea we settled on is packing bundles of DTB files into PE executables, that are loaded and signature verified by the UKI Stub. They're versioned with the kernel. UKI Stub then parses a table of contents in a PE section, and loads the right DTB and boots with it.

[lumag]

@AdrianVovk this sounds like an interesting idea. Do you have a pointer to the DTB selection code? Unfortunately Debian / Ubuntu do not use UKI images on ARM64, but maybe we can look for a close enough solution.

[TravMurav]

I think loading drivers from shim might enlarge possible attack surface.

Here I'm assuming that the shim only loads signed (and thus explicitly trusted) so I don't think it's a big problem. And as I said grub/sd-boot could as well load the driver themselves instead of the shim.

Devicetree loading should be restricted if SB is enabled.

I fully agree, this is why my dtbloader already implements (simplified) hash check, and I do agree that a better solution could be implemented.

The idea we settled on is packing bundles of DTB files into PE executables [...]

I'd strongly insist that the signature blob is detached from dtbs, and dtbs are stored as-is in a canonical tree structure (i.e. as installed by linux build system). This will allow simplified bootloaders without SB still use those.

Another part of the difficulty is that DTB isn't necessary distro agnostic.

Indeed, I've briefly thought of that and my tentative proposal is as follows:

1. Extend uapi bootloader spec with devitectreedir key (like in u-boot extlinux.conf) that would be defined as a base path where bootloader loads device dtb from. The dir is expected to contain dtbs with canonical paths (as linux installs them) and picking a file is implementation-defined.
2. Extend EFI_DT_FIXUP_PROTOCOL to provide GetCanonicalFdtName() which would return the filename of the dtb of detected device; Update u-boot and dtbloader implementations of the protocol to provide this function.
3. Implement sd-boot/grub fdtdir support using the protocol.

Then we can add some dtb verification step on top, i.e. by using a detached signature block in some dtbs/dtbsig.efi that would be a signed data-only PE. The simplified blspec implementations can ignore it, and proper (like sd-boot) will make use of it if SB is enabled.

So i.e. with sd-boot the flow would be as follows:

1. Shim verifies and boots sd-boot
2. sd-boot verifies and loads dtbloader
3. dtbloader detects the device, installs EFI_DT_FIXUP_PROTOCOL, and either installs (fallback) dtb or bails out without installing any.
4. sd-boot sees devicetreedir dtbs-6.15/ in the conf file, calls the GetCanonicalFdtName() and assembles the full path.
5. sd-boot loads the dtb, if SB is enabled, it also loads dtbs-6.15/dtbsig.efi using the shim protocol, reads the hashes and matches with the loaded dtb. The verification could also be delegated to i.e. dtbloader via a new efi protocol since dtbloader would need that code too anyway.
6. (already exists) sd-boot calls EFI_DT_FIXUP_PROTOCOL.Fixup() and dtbloader patches the newly loaded dtb with its fixups (partial-goods, panel compatible, touchpad hid id, bt/wifi mac address, etc)
7. (already exists) sd-boot boots linux

This way we can keep the boot requirements simple when SB is not used (so kernel hackers don't need to sign their dtbs all the time for example, or alternative non-uefi implementations of blspec could exist) while providing hardening with signatures for all dtbs in the SB case.

What do you think?

[AdrianVovk]

No code to point that I'm aware of. This is just a discussion we had w/ Caleb.

We'd probably want to use the same CHID algorithm as DtbLoader, with the addition of a even more generic CHID that's just a Manufacturer. (Actually this exists in DtbLoader too, it's just not documented)

Next to the UKI you'd have an drop-in dir, similar to UKI addon dirs. The drop-in dir would contain these bundles. Each file is named after a CHID. The most specific matching CHID is picked and that bundle is loaded from disk and signature verified. Inside, there's a table of contents section that contains a map of CHID -> PE section name. Possibly more info - like maybe a bit if it's a DTB or DTBO

So on the ESP would be laid out like so (but instead of writing CHID UUIDs I'll just write text in square brackets)

/kernel.efi
/kernel.efi.addon.d
     <UKI addons>
/kernel.efi.credential.d
     <Kernel-specific credentials>
/kernel.efi.dtb.d
     [HP].dtbb
     [Acer].dtbb
     [Microsoft].dtbb
     ....

Then each DTB bundle file would look like (pseudocode showing structure):

.dtbbtoc
    # CHID, Section, IsOverlay
    [HP&Envy&x360-123] aaaa false
    [HP&Envy&x360-456] aaab false
    [HP&Envy&x360-457&Sku2] aaac true
    ...

.aaaa
    <DTB for HP Envy x360-123>

.aaab
    <DTB for HP Envy x360-456>

.aaac
    <DTBO that's applied for HP Envy x360-456 sku 2>

Idk the exact handling of DTBOs because I don't know what they're used for in practice.

Also: for debugging purposes, we'd define a special place you can drop in a DTB or any number of DTBOs to be applied unconditionally (and override all this lookup logic) but only if secure boot is off. Something like dtb-override/ on the ESP.

[AdrianVovk]

Where would DtbLoader or UBoot get the canonical path of the devicetree file from?

I suppose you could make a device-specific build for each variant of device, hard compiled with the right name. But what about DtbLoader?

Other than that, seems fine to me at first glance. Type 2 BLS entries would just have an implicit dtbdir dropped next to them like I describe. Type 1 would have to configure it like you describe.

[AdrianVovk]

There's always the option of having the kernel build system produce a different directory layout at install time. Maybe it can name the DTBs after CHIDs? Then drop the canonical name stuff and let the bootloader autodiscover the right blob to load

[TravMurav]

Where would DtbLoader or UBoot get the canonical path of the devicetree file from?

Where would your build system get the chids for the dtbb files from?

my dtbloader keeps a database of chid table -> device description(dtb name) and matches using chid of the board.

Collection: https://github.com/TravMurav/dtbloader/tree/main/scripts/hwids example of hw description: https://github.com/TravMurav/dtbloader/blob/main/src/devices/acer_aspire1.c

There's always the option of having the kernel build system produce a different directory layout at install time.

One of my goals is to keep backwards compatiblility with some unusual non-uefi bootloaders like lk2nd (which currently implements u-boot extlinux with fdtdir exactly how described above, but there is no SB or uefi that could possibly be used on devices that use lk2nd to boot linux)

[lumag]

I think loading drivers from shim might enlarge possible attack surface.

Here I'm assuming that the shim only loads signed (and thus explicitly trusted) so I don't think it's a big problem. And as I said grub/sd-boot could as well load the driver themselves instead of the shim.

If I remember correctly, grub limits the set of modules that can be loaded with the SB being enforced. Also the main question is not only in loading in signed binaries, but also in failing to load revoked binaries (think about SBAT and dbx)

Devicetree loading should be restricted if SB is enabled.

I fully agree, this is why my dtbloader already implements (simplified) hash check, and I do agree that a better solution could be implemented.

Hash check just makes sure that user knows what they are doing.

The idea we settled on is packing bundles of DTB files into PE executables [...]

I'd strongly insist that the signature blob is detached from dtbs, and dtbs are stored as-is in a canonical tree structure (i.e. as installed by linux build system). This will allow simplified bootloaders without SB still use those.

I have mixed feelings here. I support using the external signatures, but I'm not so sure about the tree structure. Doing so requires patching DtbLoader for each and every platform that gets added. Maybe a better alternative is to use the DMI-like structure, e.g.:

- LENOVO/
  - 81F1.dtb 
  - 81JL.dtb
  - 81JL/
    - some-sku.dtb
  - 82AK.dtb

Another part of the difficulty is that DTB isn't necessary distro agnostic.

Indeed, I've briefly thought of that and my tentative proposal is as follows:

1. Extend [uapi bootloader spec](https://uapi-group.org/specifications/specs/boot_loader_specification/) with `devitectreedir` key (like in u-boot extlinux.conf) that would be defined as a base path where bootloader loads device dtb from. The dir is expected to contain dtbs with canonical paths (as linux installs them) and picking a file is implementation-defined.

I'd prefer for the dtb to go to the ESP. If there is a new DTB file, it can get updated. However the old DTB files per contract must continue to work with newer kernels (otherwise it's a bug which can and must be fixed).

2. Extend [EFI_DT_FIXUP_PROTOCOL](https://github.com/U-Boot-EFI/EFI_DT_FIXUP_PROTOCOL) to provide `GetCanonicalFdtName()` which would return the filename of the dtb of detected device; Update u-boot and dtbloader implementations of the protocol to provide this function.

3. Implement sd-boot/grub fdtdir support using the protocol.

Then we can add some dtb verification step on top, i.e. by using a detached signature block in some dtbs/dtbsig.efi that would be a signed data-only PE. The simplified blspec implementations can ignore it, and proper (like sd-boot) will make use of it if SB is enabled.

I think the proposed flow won't play well with the GRUB bootloader. It can chainload the EFI app, but I'm not sure you can load EFI drivers from GRUB.

So i.e. with sd-boot the flow would be as follows:

1. Shim verifies and boots sd-boot

2. sd-boot verifies and loads dtbloader

3. dtbloader detects the device, installs EFI_DT_FIXUP_PROTOCOL, and either installs (fallback) dtb or bails out without installing any.

4. sd-boot sees `devicetreedir dtbs-6.15/` in the conf file, calls the `GetCanonicalFdtName()` and assembles the full path.

GRUB developers indicated that they don't want to load DTB, especially if the SB is enabled. So it must be handled before GRUB starts (or from a very specific GRUB module, which doesn't really make sense).

5. sd-boot loads the dtb, if SB is enabled, it also loads `dtbs-6.15/dtbsig.efi` using the shim protocol, reads the hashes and matches with the loaded dtb. The verification could also be delegated to i.e. dtbloader via a new efi protocol since dtbloader would need that code too anyway.

It definitely doesn't make sense to make each and every bootloader implement DTB verification. So, I think, DtbLoader (or shim) should load, verify and install DTB if it can find one.

6. (already exists) sd-boot calls `EFI_DT_FIXUP_PROTOCOL.Fixup()` and dtbloader patches the newly loaded dtb with its fixups (partial-goods, panel compatible, touchpad hid id, bt/wifi mac address, etc)

Not implemented by grub, but most likely it should be done. I like your MAC address fixups. The rest of the items should be handled on a DTB level before loading.

7. (already exists) sd-boot boots linux

This way we can keep the boot requirements simple when SB is not used (so kernel hackers don't need to sign their dtbs all the time for example, or alternative non-uefi implementations of blspec could exist) while providing hardening with signatures for all dtbs in the SB case.

What do you think?

[lumag]

my dtbloader keeps a database of chid table -> device description(dtb name) and matches using chid of the board.

This means that for each new laptop we have to patch the dtbloader, get updated version into the distro and then make the distro authors sign it. I think it's too much trouble.

One of my goals is to keep backwards compatiblility with some unusual non-uefi bootloaders like lk2nd

I think those are separate topics as the bootflows are completely different. ABL loads the dtb from the Android image / dtbo partition, X13s loads the named dtb from ESP, RB3gen2 loads 'combined-dtb' from ESP, sd-boot might load it from /usr/lib/linux-image-$(uname -r), lk2nd might load it from some other place. EFI DtbLoader can have its own useful storage format.

[TravMurav]

failing to load revoked binaries

which would work properly if dtbloader or dtbsig is in dbx: uefi or shim would refuse to load an efi in dbx wouldn't it?

Hash check just makes sure that user knows what they are doing.

As I said, this is existing //temporary// solution. It should be replaced with something better (my proposal described above)

Doing so requires patching DtbLoader for each and every platform that gets added.

I don't see this as a problem. You need to collect DMI/chids in any case; and between new device appearing in linux-next and new kernel being released there will be at least 6 weeks to add a trivial patch in dtbloader (it's fully scripted, one cli invocation on the device in question). Letting distros go loose and collect those separately each in their own place will leave us with a mess where no one has a unified map of chid->device.

Maybe a better alternative is to use the DMI-like structure

I want to have generic images that could be loaded both on UEFI and on non-uefi (non smbios) targets. This would require having multiple dtb trees.

... I'm not sure you can load EFI drivers from GRUB.

I assume this is not impossible to implement though.

... bootflows are completely different.

I wish not to unify everything but allow non-uefi bootloaders to /be able/ to implement "type 2 BLS" without assuming there is UEFI or smbios.

[TravMurav]

and then make the distro authors sign it. I think it's too much trouble.

except they would still need to sign dtbs for each kernel release which is even more often than new laptops being added to linux.

[AdrianVovk]

I'd prefer for the dtb to go to the ESP. If there is a new DTB file, it can get updated. However the old DTB files per contract must continue to work with newer kernels (otherwise it's a bug which can and must be fixed).

In theory yes. In practice no.

My understanding is that there have been intentional breaks in compatibility for DTB because that was simply practical and everyone ships the DTB version locked anyways. And I have no idea if forwards compatibility is something that's considered.

Dual-booting exists, and if you need to run ancient RHEL with some old kernel for work and then use arch Linux rolling with the latest Torvalds-git kernel you're going to run into situations where you jump between kernel versions. Plus, we're just assuming that we're talking about an upstream kernel here...

[AdrianVovk]

Just to clarify,

I'd prefer for the dtb to go to the ESP

The directory would be in the ESP. We'd just define there where in the ESP to look for it via the setting in BLS Type #1

[lumag]

failing to load revoked binaries

which would work properly if dtbloader or dtbsig is in dbx: uefi or shim would refuse to load an efi in dbx wouldn't it?

I'm not sure that we can or are supposed to get MoK / distro-signed binaries into DBX. See how this is handled by SBAT, which, to my understanding, provides some revocation capabilities. But then again the issue is that currently it's expected to have a small list of items.

Hash check just makes sure that user knows what they are doing.

As I said, this is existing //temporary// solution. It should be replaced with something better (my proposal described above)

Yes, I know. Just wanted to explicitly point out that it's not suitable for SB case.

Doing so requires patching DtbLoader for each and every platform that gets added.

I don't see this as a problem. You need to collect DMI/chids in any case; and between new device appearing in linux-next and new kernel being released there will be at least 6 weeks to add a trivial patch in dtbloader (it's fully scripted, one cli invocation on the device in question). Letting distros go loose and collect those separately each in their own place will leave us with a mess where no one has a unified map of chid->device.

A map (or a script) is fine. We can have a separate repo or we can have it inside linux-kernel, if nothing else. But resigning bootloader to support new devices doesn't sound correct.

Maybe a better alternative is to use the DMI-like structure

I want to have generic images that could be loaded both on UEFI and on non-uefi (non smbios) targets. This would require having multiple dtb trees.

The ESP will get several DTB files, that's it. I don't see it as a trouble. It doesn't need to get all existing DTB files, only those for the UEFI-but-not-DTB platforms. If you are reflashing the whole image it's even less trouble as everything gets updated at the same time.

... I'm not sure you can load EFI drivers from GRUB.

I assume this is not impossible to implement though.

... bootflows are completely different.

I wish not to unify everything but allow non-uefi bootloaders to /be able/ to implement "type 2 BLS" without assuming there is UEFI or smbios.

Heh. UKI. My world is still stuck with abootimg on some platforms, Image on other targets and some hacks on the old obscure platforms.

and then make the distro authors sign it. I think it's too much trouble. except they would still need to sign dtbs for each kernel release which is even more often than new laptops being added to linux.

The kernel and modules gets signed anyway, so signing DTB will be just another step in the same kernel-signing process.

[lumag]

My understanding is that there have been intentional breaks in compatibility for DTB because that was simply practical and everyone ships the DTB version locked anyways. And I have no idea if forwards compatibility is something that's considered.

Yes. It is considered pretty significantly (at least on the arm64/qcom platforms, upstream world). We keel support for legacy bindings, legacy drivers, etc. You can check the git history of drivers/phy/qualcomm/phy-qcom-qmp-* which serves one of examples.

[lumag]

Likewise panel-edp.c keeps support for the panels that could have ever be used via the old (non-aux-bus) bindings for the sake of being able to boot with the old DTB.

[AdrianVovk]

Maybe the CHID->filename mapping table can live in the detached signature file instead of DtbLoader. The issue is that DtbLoader is kinda supposed to be a cross-distro resource - there's really no reason to have multiple DtbLoaders floating around, right?

So similar to the "single shim" idea that's being worked towards (AFAIK) DtbLoaders could be a cross-distro signed thing that distros don't have to deal with other than making sure their boot path loads the driver

Then distros provide their own DTBs. Mapping table of CHID->DTB lives upstream in the kernel. Detached signature file includes it, so the bootloader can find the DTB, verify it, and include it in the boot chain.

[lumag]

I was thinking about a per-dtb signatures, just pkcs7, rather than a single file with multiple signatures. A signed DB makes sense, but then we need to parse it. Doing several directory walks sounds easier and safer.

[AdrianVovk]

Does UEFI provide a way to check the signature of something that isn't a PE binary? I was under the impression that it doesn't.

Otherwise I presume individual DTB files can be wrapped in PE headers and signed that way too...

[lumag]

Does UEFI provide a way to check the signature of something that isn't a PE binary? I was under the impression that it doesn't.

I think it doesn't. Last time I needed something close, I have been doing detached PKCS7

Otherwise I presume individual DTB files can be wrapped in PE headers and signed that way too...

Well, in the worst case we can bundle a set of DTB files together with some form of database as a single PE file, sign it and make the DtbLoader dissect that after verifying the signature through the existing protocol.

[lumag]

@vathpela @mjg59 @martinezjavier @lcp @jsetje @steve-mcintyre @frozencemetery (just git-grepped for committers)

Can we please hear a word from you? We can continue our discussions here, if it sounds suitable for shim to load DTBs or move to some other place (if it is not). But first let's settle whether shim is a proper place for it. Maybe we can discuss it at LPC (but I'm not sure who is going to come).

Some kind of summary of proposals:

• Make shim load, verify and install the DTB
• Make shim load (signed) drivers, one of the drivers can load, verify and install the DTB
• Make bootloaders load the driver that loads, verifies and installs the DTB
• Make bootloaders load, verify and use the DTB on their own

For all the cases except the first one we also need one of:

• Extend shim protocol with support for verifying PKCS7 detached signatures against distro key / MOK
• Extend shim protocol with support for verifying single file against detached signature in a PE file
• Just wrap DTBs in PE, one-to-one
• Just wrap all DTBs in a single PE file

[robclark]

My understanding is that there have been intentional breaks in compatibility for DTB because that was simply practical and everyone ships the DTB version locked anyways. And I have no idea if forwards compatibility is something that's considered.

A new dtb with older kernel should always work. And honestly, enforcing that wouldn't be a bad thing.

Ideally basic ACPI boot would work enough to install a distro and fwupd (with not-yet written fwupd-dtb plugin), which would then download the latest dtb "firmware" (and DtbLoader.efi?) and drop them in the ESP. (Basic ACPI boot for installer and then grab latest dtb from linux kernel seems to be the approach freebsd is taking on x1 laptops.. I'm not sure what is missing for basic ACPI boot on linux)

[lumag]

New DTB with older kernels might break, there is no guarantee for that. Missing drivers, missing compatibles, etc. On the other hand old DTB with new kernels should work in most of the cases.

For the ACPI I think we are missing bits and pieces here and there. For example to enable the DWC3-multiport support Linux dropped ACPI support from the dwc3-qcom driver. I'd say nobody uses it, nobody tests it, so there can be a lot of things that might not fully work.

Also, as I wrote, the "grab the devicetree" doesn't play well with the SecureBoot support. Modifying EFI vars after booting Linux is also impossible on Miix-630 and Yoga-C630 (shim can handle that).

[robclark]

New DTB with older kernels might break, there is no guarantee for that. Missing drivers, missing compatibles, etc. On the other hand old DTB with new kernels should work in most of the cases.

hmm, we pretty much need this not to break if we want to be able to roll back to an older kernel

For the ACPI I think we are missing bits and pieces here and there. For example to enable the DWC3-multiport support Linux dropped ACPI support from the dwc3-qcom driver. I'd say nobody uses it, nobody tests it, so there can be a lot of things that might not fully work.

If we don't have basic ACPI boot support, then dt laptops will always be second class citizens which don't have linux support (for end users) until ~6mo after they are released :-/

IMO it is pretty important that basic ACPI boot works, at least enough to install and check for updates (possibly over usb-c eth adapter)

Also, as I wrote, the "grab the devicetree" doesn't play well with the SecureBoot support.

As long as whatever installs the dtb also installs signature, why would this be an issue?

Modifying EFI vars after booting Linux is also impossible on Miix-630 and Yoga-C630 (shim can handle that).

That is probably ok.. no one is making more of these devices, and the folks who run linux on them thus far have figured out other ways to load dtb.

[AdrianVovk]

Extend shim protocol with support for verifying PKCS7 detached signatures against distro key / MOK

I suspect that this would not fly with systemd-boot. Something sd-boot has always been doing is ensuring all functionality works in bare UEFI. So if something is a shim exclusive feature we can't justify using it because we expect many deployments without shim. Those deployments still need devicetrees.

[lumag]

New DTB with older kernels might break, there is no guarantee for that. Missing drivers, missing compatibles, etc. On the other hand old DTB with new kernels should work in most of the cases.

hmm, we pretty much need this not to break if we want to be able to roll back to an older kernel

Yes. However, after the initial period of instability the DTB is usually pretty stable, there is little pressure to update it.

For the ACPI I think we are missing bits and pieces here and there. For example to enable the DWC3-multiport support Linux dropped ACPI support from the dwc3-qcom driver. I'd say nobody uses it, nobody tests it, so there can be a lot of things that might not fully work.

If we don't have basic ACPI boot support, then dt laptops will always be second class citizens which don't have linux support (for end users) until ~6mo after they are released :-/

IMO it is pretty important that basic ACPI boot works, at least enough to install and check for updates (possibly over usb-c eth adapter)

I know. I hope we can get back to it at some point.

Also, as I wrote, the "grab the devicetree" doesn't play well with the SecureBoot support.

As long as whatever installs the dtb also installs signature, why would this be an issue?

Well, only if it gets installed for some kind of DtbLoader. GRUB disables "devicetree" command if SB is enabled.

Modifying EFI vars after booting Linux is also impossible on Miix-630 and Yoga-C630 (shim can handle that).

That is probably ok.. no one is making more of these devices, and the folks who run linux on them thus far have figured out other ways to load dtb.

I fear that there can be more devices with such an issue. E.g. I wouldn't be surprised if the new RB3gen2 IoT board has the same issue (note, this is a pure speculation, I didn't have a chance to verify whether it works or not).

[robclark]

New DTB with older kernels might break, there is no guarantee for that. Missing drivers, missing compatibles, etc. On the other hand old DTB with new kernels should work in most of the cases.

hmm, we pretty much need this not to break if we want to be able to roll back to an older kernel

Yes. However, after the initial period of instability the DTB is usually pretty stable, there is little pressure to update it.

hmm, maybe there should be a way to mark dtb as "staging" early on, so they are not included in DtbLoader initially?

For the ACPI I think we are missing bits and pieces here and there. For example to enable the DWC3-multiport support Linux dropped ACPI support from the dwc3-qcom driver. I'd say nobody uses it, nobody tests it, so there can be a lot of things that might not fully work.

If we don't have basic ACPI boot support, then dt laptops will always be second class citizens which don't have linux support (for end users) until ~6mo after they are released :-/ IMO it is pretty important that basic ACPI boot works, at least enough to install and check for updates (possibly over usb-c eth adapter)

I know. I hope we can get back to it at some point.

Also, as I wrote, the "grab the devicetree" doesn't play well with the SecureBoot support.

As long as whatever installs the dtb also installs signature, why would this be an issue?

Well, only if it gets installed for some kind of DtbLoader. GRUB disables "devicetree" command if SB is enabled.

It would be nice to have a solution that was a bit distro-agnostic, since (for ex) freebsd also needs a way to pick/load the correct dtb

Modifying EFI vars after booting Linux is also impossible on Miix-630 and Yoga-C630 (shim can handle that).

That is probably ok.. no one is making more of these devices, and the folks who run linux on them thus far have figured out other ways to load dtb.

I fear that there can be more devices with such an issue. E.g. I wouldn't be surprised if the new RB3gen2 IoT board has the same issue (note, this is a pure speculation, I didn't have a chance to verify whether it works or not).

Hmm, well I guess setting EFI vars is only something that needs to be done when installing DtbLoader. Perhaps for non-consumer devices which have a more limited UEFI implementation, users must do this step manually.

[anonymix007]

I stumbled across the the fact that while UKI spec does allow to have multiple DTBs, there's just no way to make any selection between them. Here's my attempt to implement that selection in sd-stub: https://github.com/systemd/systemd/pull/34158 (absolutely untested, but seems to at least compile in CI)

It is heavily based on dtbloader, but instead of matching against the dtb file name, it actually uses compatible and selects the appropriate ".dtb" PE section from UKI based on that.

This solves a few problems brought up here: SecureBoot (as UKIs can be signed), paths (as they will be used only during the UKI build and can be whatever) and the need to have DTBs in ESP (I actually run UKI from XBOOTLDR partition without any issues. Non-UKI kernel will just fail with "EFI stub: ERROR: Exit boot services failed" and it's basically the main reason why I even tried UKIs).

This will allow to make a generic maybe even signed for secure boot X1E/8cxg3/... kernel image which can be just used by any distro without the need to make users install efi drivers.

[lumag]

@anonymix007 that's a good point, thank you. However some distros don't use UKI, so the DTB issue is still valid.

[anonymix007]

@lumag then these distros should look into starting to use UKI. Isn't the current situation is that secure boot just doesn't play nicely with DTB loading? If so, something clearly needs to be changed. It's basically a matter of setting a few options in mkinitcpio config file (+ ukify config) and directing the bootloader to use it instead (at least this did it for me on ArchLinux ARM).

[lumag]

@anonymix007 initramfs-tools here. The questions of switching the whole distro to UKI is bigger and more troublesome than the question of DTB handling.

[AdrianVovk]

You don't have to use all the features of a UKI. You can make a UKI of just the kernel and devicetrees. Then treat that as a normal Linux kernel image in Grub, and boot everything else like you always have

[robclark]

Getting a bit off topic, but wouldn't UKI require signing to be on-device (since initrd is embedded in UKI)? Or has that been solved since I last looked at UKIs?

[anonymix007]

DTB issue is still valid

Actually, I haven't said that it's not valid. What I said is that it can be solved with UKI and the proposed sd-stub patch.

initramfs-tools

You probably may add hooks to run systemd-ukify after the initramfs was created. For kernels to be signed this can't be done on client machines anyway, so even if there's absolutely no way to add a hook, this isn't an issue for secure boot.

The questions of switching the whole distro to UKI is bigger and more troublesome than the question of DTB handling.

Not really the whole, but rather just platforms which require DTB to boot. Either way, such a point needs some evidences. What are the other options? Forcing user to install an EFI driver? It's already quite complicated, no need to make it even harder for users. Adding intermediate EFI loader which do the dtb selection and then chainload grub? Well, this is basically what UKI is (except without grub, just the kernel directly), just as a few separate files. I don't see how converting to that solution (which doesn't yet even exist) is less complicated.

[anonymix007]

@robclark AFAIU, distros can just package UKI with pregenerated initrd instead of leaving it to the device. There's really no way around this if it needs to be verified.

[julian-klode]

Like the kernel the UKI needs to be signed by the distribution obviously, or you enroll a key in MOK and build your own.

[robclark]

@robclark AFAIU, distros can just package UKI with pregenerated initrd instead of leaving it to the device. There's really no way around this if it needs to be verified.

Right, but that makes the initrd huge, especially on arm+dtb where all the clk/power/interconnect drivers are in kernel, and different per SoC (instead of all hidden in ACPI). Compare arm64 vs x86 devconfigs if you don't believe me, IIRC the arm64 one is 5x bigger.

Unless distro's are building per-device UKI images, this doesn't seem like a great solution.

[anonymix007]

There isn't a better solution though: if initrd has to be signed, it has to be provided by distros.

sd-boot will have a concept of profiles (https://github.com/systemd/systemd/pull/33512), so maybe it'll be enough to have a per-device (more like per-platform though, there aren't that much i.e. WoA devices anyway) add-on with initrd.

[AdrianVovk]

Again: You don't need to include the initrd if you don't want to. It's an optional section. Make a UKI without an initrd and treat it like any other kernel image. The only difference is now that it can secure-load DeviceTrees (or even just have them embedded). If you want more UKI features later you can start using them.

If that doesn't work today, it's a bug IMO. The spec says that it should - the initrd section is optional in the spec.

[lumag]

Generally I think that while it might be recommended to use UKI, we should not be imposing this as a requirement.

[robclark]

Generally I think that while it might be recommended to use UKI, we should not be imposing this as a requirement.

Also, remember that OpenBSD (and perhaps others) are using dtb, and I suspect would like to be able to re-use the same mechanism for picking/loading dtb.