Add configuration option to boot an alternative 2nd stage

[esnowberg]

Add runtime option to load a different 2nd stage boot loader.

Most projects working with Unified Kernel Images (UKI) have one thing in common. If they want to use a distro signed shim to boot it, they have to rename their UKI to grub{arch}.efi. Having a UKI named grub can be confusing.

Add the ability for shim to load an optional configuration file. This new file is called "options.csv". The configuration file is completely optional. If used, it is located in the same directory as the booted shim. The "options.csv" file currently allows a single entry. Other options could be added to it in the future. The first and only entry in the file is the name of the secondary boot loader shim will load. The "options.csv" file is in Unicode LE format.

This allows a signed shim to directly load a UKI without the need to rename it to grub. Shim's transitive trust is maintained. If the alternative 2nd stage can not be verified, it will not boot.

Example on how to create the options.csv file:

echo "uki.efi," | iconv -f us-ascii -t utf-16le > options.csv

Unlike the RFE request here: https://github.com/rhboot/shim/issues/573 a csv file is being used instead of walking through the directory. Shim already has precedents for loading a csv file. Also, walking an ESP directory has shown to have problems when network booting with shim.

[jsetje]

Since creating that file requires the same permissions to access the system partition that renaming grubx64.efi would this does not change what a user or distro can do. This is just a clean, extensible interface for an existing capability.

[bluca]

walking an ESP directory has shown to have problems when network booting with shim.

If that is the issue with discovering what's available, maybe there could be a build-time option to disable it for systems where things are that broken? Both sd-stub and sd-boot walk the ESP in various places, so if you are using either a UKI or sd-boot you'd already be out of luck on such systems.

Maybe an alternative approach: look for grub first at the fixed path, with no filesystem spelunking at first - and only if not found iterate over the directory content? That way if you are using shim+grub there is no change in functionality nor operations, and if you use shim+uki or shim+sd-boot then you'll already get filesystem spelunking in the second stage so there would be no difference.

[esnowberg]

walking an ESP directory has shown to have problems when network booting with shim.

If that is the issue with discovering what's available, maybe there could be a build-time option to disable it for systems where things are that broken? Both sd-stub and sd-boot walk the ESP in various places, so if you are using either a UKI or sd-boot you'd already be out of luck on such systems.

Here is a reference to the issue: https://github.com/rhboot/shim/issues/482

Distros use a single build-time option for their shim.

Maybe an alternative approach: look for grub first at the fixed path, with no filesystem spelunking at first - and only if not found iterate over the directory content? That way if you are using shim+grub there is no change in functionality nor operations, and if you use shim+uki or shim+sd-boot then you'll already get filesystem spelunking in the second stage so there would be no difference.

Where does systemd-stub do directory discovery during boot? How does systemd-stub solve this when booting from TFTP, HTTP, or HTTPS?

[bluca]

Maybe an alternative approach: look for grub first at the fixed path, with no filesystem spelunking at first - and only if not found iterate over the directory content? That way if you are using shim+grub there is no change in functionality nor operations, and if you use shim+uki or shim+sd-boot then you'll already get filesystem spelunking in the second stage so there would be no difference.

Where does systemd-stub do directory discovery during boot? How does systemd-stub solve this when booting from TFTP, HTTP, or HTTPS?

At least for addons, type 1 configs, drivers, some of these happen in the stub, some in sd-boot. I don't believe there's any particular care taken for network booting, the filesystem protocol is used.

[esnowberg]

Maybe an alternative approach: look for grub first at the fixed path, with no filesystem spelunking at first - and only if not found iterate over the directory content? That way if you are using shim+grub there is no change in functionality nor operations, and if you use shim+uki or shim+sd-boot then you'll already get filesystem spelunking in the second stage so there would be no difference.

Where does systemd-stub do directory discovery during boot? How does systemd-stub solve this when booting from TFTP, HTTP, or HTTPS?

At least for addons, type 1 configs, drivers, some of these happen in the stub, some in sd-boot. I don't believe there's any particular care taken for network booting, the filesystem protocol is used.

The solution in this pull request is general enough to get to the second stage without the need to do directory discovery. It would be up to the second stage to determine if it wants to support network booting or not. Why place a limitation on shim if the next stage currently doesn’t support something?

[julian-klode]

I think this is better served by the changes to integrate fallback loader and a patch to disable the creation of boot entries for the fallback loader ones.

[bluca]

Maybe an alternative approach: look for grub first at the fixed path, with no filesystem spelunking at first - and only if not found iterate over the directory content? That way if you are using shim+grub there is no change in functionality nor operations, and if you use shim+uki or shim+sd-boot then you'll already get filesystem spelunking in the second stage so there would be no difference.

Where does systemd-stub do directory discovery during boot? How does systemd-stub solve this when booting from TFTP, HTTP, or HTTPS?

At least for addons, type 1 configs, drivers, some of these happen in the stub, some in sd-boot. I don't believe there's any particular care taken for network booting, the filesystem protocol is used.

The solution in this pull request is general enough to get to the second stage without the need to do directory discovery. It would be up to the second stage to determine if it wants to support network booting or not. Why place a limitation on shim if the next stage currently doesn’t support something?

Because extra config files means extra moving parts, and also extra things that have to be measured. It's much nicer to just do the right thing.

[esnowberg]

Because extra config files means extra moving parts, and also extra things that have to be measured. It's much nicer to just do the right thing.

With the changes currently in this pull request, would the config file really need to be measured? Currently if someone renames their UKI to grub, this should already be handled in the current measurement.

In the future if a new option were added, it may be determined if measurement on the config would be appropriate. But I don’t see why it would be needed at this time.

[bluca]

Because extra config files means extra moving parts, and also extra things that have to be measured. It's much nicer to just do the right thing.

With the changes currently in this pull request, would the config file really need to be measured? Currently if someone renames their UKI to grub, this should already be handled in the current measurement.

In the future if a new option were added, it may be determined if measurement on the config would be appropriate. But I don’t see why it would be needed at this time.

It's an (unsigned) artifact that changes the behaviour of the first stage loader, so that sounds like something that would need to be measured. Besides that, and other than config files being a pain to handle, we also rename UKIs for boot counting, having to keep a config file in sync with that would make it extra fragile and complicated.

[esnowberg]

Because extra config files means extra moving parts, and also extra things that have to be measured. It's much nicer to just do the right thing.

With the changes currently in this pull request, would the config file really need to be measured? Currently if someone renames their UKI to grub, this should already be handled in the current measurement. In the future if a new option were added, it may be determined if measurement on the config would be appropriate. But I don’t see why it would be needed at this time.

It's an (unsigned) artifact that changes the behaviour of the first stage loader, so that sounds like something that would need to be measured.

If it is determined appropriate to measure the config file, a follow on patch for shim could do that. No other entity other than shim would need to be responsible for doing the measurement.

Besides that, and other than config files being a pain to handle, we also rename UKIs for boot counting, having to keep a config file in sync with that would make it extra fragile and complicated.

The intention of this change was to add a general purpose way for any secondary boot loader to be used with a signed shim, while maintaining shim’s transitive trust. This is not geared specifically for sd-boot but any secondary boot loader.

It would be good to hear from the shim maintainers on their opinion of adding new code to shim that does discovery on a directory. It was my believe that adding new code that did this would be a no-go.

[bluca]

The intention of this change was to add a general purpose way for any secondary boot loader to be used with a signed shim, while maintaining shim’s transitive trust. This is not geared specifically for sd-boot but any secondary boot loader.

Sure, I don't know how many other second stages for uefi + sb are out there, but if this cuts out, or at the very least makes it unnecessarily difficult and awkward to use, with sd-stub based UKIs or sd-boot, then that should be a concern, as that's at the very least a good chunk of the total, no?

[bluca]

This looks fine as a starting point to me, then I can propose a follow-up PR to auto-discovery if the config file is not found. That should satisfy all constraints: on systems where fragile firmware implementations are used, one can ensure the new config file is provided, and all will be well. For systems where this is not an issue, users can choose to omit it, and do the autodiscovery. Would this work as an approach?

[bluca]

Ping - would it be possible to bring this forward?