Standardizing the devicetree overlay handling at the bootloader level

[Mani-Sadhasivam]

Embedded devices need a standard way to pass devicetree overlays during boot. Currently, both u-boot and edk2 (not sure how much) have support for handling devicetree overlays, so a standard way to pass and apply them before passing the final dtb to kernel needs to be provisioned.

Below are some of the options:

1. Separate dtbo partition which can hold an overlay at a time.
2. A vfat based dtbo partition for handling multiple overlays.
3. Multiple dtbo partitions like how Android is doing.

CC: @johnstultz-work

[robherring]

IMO, separate partition(s) is not the way to handle overlays. We already have a defined way to handle firmware files in the EFI system partition and there is nothing in particular about overlay files to handle them in a special way. What's needed here is how to define which overlays to apply at boot. That could be as simple as allowing 'devicetree' entries in boot menu files to be a list of the base dtb plus any overlays rather than just a single dtb.

[daniel-thompson]

Embedded devices need a standard way to pass devicetree overlays during boot.

Not sure if I fully accept this. There are a subset of devices (mostly development boards with expansion headers) where this is true but I don't think this is the norm for embedded devices.

In other words whilst there could be a special ESP idiom or a GUID defined to allow a capsule to install a DT overlay I'm not sure EBBR would ever want to require an implementation to provide it (I'm not even sure if would merit a recommendation).

Note also that with EBBR as it is today the OS is already free to interfere with the hardware description if it wishes. For example an OS bootloader, such as grub, could implement a dtoverlay command to modify the device tree passed by the firmware before the kernel boots. A mechanism standardised using this approach would perhaps not be in scope for EBBR since it would be implemented purely within the OS.

[fozog]

What about a maintenance boot?

The uart may be excluded from dt for normal boot, the maintenance personnel may be offered an alternate boot option that would add the uart as an overlay ?

What about EBBR VMs? (Static partitioning or traditional VMs). There would be many reasons to overlay hardware definitions.

Le 12 avr. 2019 à 16:18, Daniel Thompson notifications@github.com a écrit :

Embedded devices need a standard way to pass devicetree overlays during boot.

Not sure if I fully accept this. There are a subset of devices (mostly development boards with expansion headers) where this is true but I don't think this is the norm for embedded devices.

In other words whilst there could be a special ESP idiom or a GUID defined to allow a capsule to install a DT overlay I'm not sure EBBR would ever want to require an implementation to provide it (I'm not even sure if would merit a recommendation).

Note also that with EBBR as it is today the OS is already free to interfere with the hardware description if it wishes. For example an OS bootloader, such as grub, could implement a dtoverlay command to modify the device tree passed by the firmware before the kernel boots. A mechanism standardised using this approach would perhaps not be in scope for EBBR since it would be implemented purely within the OS.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[trini]

Embedded devices need a standard way to pass devicetree overlays during boot.

Not sure if I fully accept this. There are a subset of devices (mostly development boards with expansion headers) where this is true but I don't think this is the norm for embedded devices.

I strongly disagree. We're well past a decade now where production-revB and production-revA are handled with two device trees where the delta is some tiny amount that could be handled with an overlay or (no, not for a decade now, but..) is handled with an overlay. There's also "new overlay as previous display is no longer available".

[daniel-thompson]

On Fri, Apr 12, 2019 at 08:24:52AM -0700, Tom Rini wrote:

Embedded devices need a standard way to pass devicetree overlays during boot.

Not sure if I fully accept this. There are a subset of devices (mostly development boards with expansion headers) where this is true but I don't think this is the norm for embedded devices.

I strongly disagree. We're well past a decade now where production-revB and production-revA are handled with two device trees where the delta is some tiny amount that could be handled with an overlay or (no, not for a decade now, but..) is handled with an overlay. There's also "new overlay as previous display is no longer available".

I'm not disputing the use of overlays; it was the "need a standard way to pass DT overlays during boot" which, at least to me, implied some kind of interface accessible to the OS.

Rev A versus rev B is useful but ideally also handled in the boot firmware and hooked up to some board revision auto-detection code. Other UI features to tweak the features or boot modes can also be useful but are also strictly within the firmware implementation boundary.

What is that motivates standardizing how firmware internally uses overlays between, for example, EDK2 and u-boot.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/ARM-software/ebbr/issues/41#issuecomment-482616274

[daniel-thompson]

On Fri, Apr 12, 2019 at 03:20:03PM +0000, fozog wrote:

What about a maintenance boot?

The uart may be excluded from dt for normal boot, the maintenance personnel may be offered an alternate boot option that would add the uart as an overlay ?

Aren't these simply firmware features. Sure, maybe they use DT overlay to implement them but what benefit does a "standard way to pass devicetree overlays during boot" bring for these use cases?

What about EBBR VMs? (Static partitioning or traditional VMs). There would be many reasons to overlay hardware definitions.

Similarly I'd expect the VM monitor to construct the DT on-the-fly but I'm not clear how this affects anything by the firmware. What does EBBR need to standardize to simplify this?

Daniel.

Le 12 avr. 2019 à 16:18, Daniel Thompson notifications@github.com a écrit :

Embedded devices need a standard way to pass devicetree overlays during boot.

Not sure if I fully accept this. There are a subset of devices (mostly development boards with expansion headers) where this is true but I don't think this is the norm for embedded devices.

In other words whilst there could be a special ESP idiom or a GUID defined to allow a capsule to install a DT overlay I'm not sure EBBR would ever want to require an implementation to provide it (I'm not even sure if would merit a recommendation).

Note also that with EBBR as it is today the OS is already free to interfere with the hardware description if it wishes. For example an OS bootloader, such as grub, could implement a dtoverlay command to modify the device tree passed by the firmware before the kernel boots. A mechanism standardised using this approach would perhaps not be in scope for EBBR since it would be implemented purely within the OS.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/ARM-software/ebbr/issues/41#issuecomment-482614565

[johnstultz-work]

IMO, separate partition(s) is not the way to handle overlays. We already have a defined way to handle firmware files in the EFI system partition and there is nothing in particular about overlay files to handle them in a special way. What's needed here is how to define which overlays to apply at boot. That could be as simple as allowing 'devicetree' entries in boot menu files to be a list of the base dtb plus any overlays rather than just a single dtb.

Some of the reasons I'd suggest to consider overlay partitions is: 1) Alignment with Android and their need to have split soc/oem/odm ownership for the soc/board/device specific components.

2) For things like mezzanine/board expansion, flashing a partition with a mezzanine specific dtbo can be much easier user-experience then trying to add a file to an existing partition.

[robherring]

IMO, separate partition(s) is not the way to handle overlays. We already have a defined way to handle firmware files in the EFI system partition and there is nothing in particular about overlay files to handle them in a special way. What's needed here is how to define which overlays to apply at boot. That could be as simple as allowing 'devicetree' entries in boot menu files to be a list of the base dtb plus any overlays rather than just a single dtb.

Some of the reasons I'd suggest to consider overlay partitions is:

1. Alignment with Android and their need to have split soc/oem/odm ownership for the soc/board/device specific components.

I think there are a lot of other problems supporting Android on an EBBR compliant system with the existing Android partitioning before you even consider overlays. We can't really support Android being at the receiving end of whatever Google makes up and vendors modify in their own unique ways. We tried to get participation with EBBR.

2. For things like mezzanine/board expansion, flashing a partition with a mezzanine specific dtbo can be much easier user-experience then trying to add a file to an existing partition.

Repartitioning every time you add another overlay is a good experience? Personally, being able to change individual files with standard tools (e.g. cp) and edit text files to control boot is a better user experience than having to deal with custom container images.

How do you define what overlays to apply?

[johnstultz-work]

1. Alignment with Android and their need to have split soc/oem/odm ownership for the soc/board/device specific components.

I think there are a lot of other problems supporting Android on an EBBR compliant system with the existing Android partitioning before you even consider overlays. We can't really support Android being at the receiving end of whatever Google makes up and vendors modify in their own unique ways. We tried to get participation with EBBR.

Was there? Because in my (admittedly sporadic) discussions with folks, "we're not looking at android yet" was a common theme. And at last plumbers there was actually a fair amount of interest from Android developers I spoke with on standardizing bootloaders. So while Andorid has a long history of doing it their own way, I don't think dismissing/ignoring them is a great way to get folks to the table here.

1. For things like mezzanine/board expansion, flashing a partition with a mezzanine specific dtbo can be much easier user-experience then trying to add a file to an existing partition.

Repartitioning every time you add another overlay is a good experience? Personally, being able to change individual files with standard tools (e.g. cp) and edit text files to control boot is a better user experience than having to deal with custom container images.

That's sort of the "pets vs cattle" difference though. On a desktop system you maintain, yes copying dtbo files over to the UEFI system partition and editing bootloader text files might make sense. But for many embedded devices, remotely flashing a provided partition via fastboot is much easier.

How do you define what overlays to apply?

It could be as simple as all the dtbo files in the partition in order by name? By using a partition there doesn't have to be a Bootloader UI involved.

[Mani-Sadhasivam]

Embedded devices need a standard way to pass devicetree overlays during boot.

Not sure if I fully accept this. There are a subset of devices (mostly development boards with expansion headers) where this is true but I don't think this is the norm for embedded devices.

In other words whilst there could be a special ESP idiom or a GUID defined to allow a capsule to install a DT overlay I'm not sure EBBR would ever want to require an implementation to provide it (I'm not even sure if would merit a recommendation).

Note also that with EBBR as it is today the OS is already free to interfere with the hardware description if it wishes. For example an OS bootloader, such as grub, could implement a dtoverlay command to modify the device tree passed by the firmware before the kernel boots. A mechanism standardised using this approach would perhaps not be in scope for EBBR since it would be implemented purely within the OS.

I kind of assumed that EBBR's scope is stretched to all components till booting the kernel. Since EBBR targets standard booting procedure, providing a standard way to load dt overlays which in turn affecting the functionality made sense to me.

[Mani-Sadhasivam]

1. Alignment with Android and their need to have split soc/oem/odm ownership for the soc/board/device specific components.

I think there are a lot of other problems supporting Android on an EBBR compliant system with the existing Android partitioning before you even consider overlays. We can't really support Android being at the receiving end of whatever Google makes up and vendors modify in their own unique ways. We tried to get participation with EBBR.

Was there? Because in my (admittedly sporadic) discussions with folks, "we're not looking at android yet" was a common theme. And at last plumbers there was actually a fair amount of interest from Android developers I spoke with on standardizing bootloaders. So while Andorid has a long history of doing it their own way, I don't think dismissing/ignoring them is a great way to get folks to the table here.

1. For things like mezzanine/board expansion, flashing a partition with a mezzanine specific dtbo can be much easier user-experience then trying to add a file to an existing partition.

Repartitioning every time you add another overlay is a good experience? Personally, being able to change individual files with standard tools (e.g. cp) and edit text files to control boot is a better user experience than having to deal with custom container images.

That's sort of the "pets vs cattle" difference though. On a desktop system you maintain, yes copying dtbo files over to the UEFI system partition and editing bootloader text files might make sense. But for many embedded devices, remotely flashing a provided partition via fastboot is much easier.

You mean, the user has to switch the board to fastboot mode everytime he wants to change the overlay? That doesn't sound like a nice UI. I agree with Rob's feedback of changing the overlay by modifying the mount partition. This also aligns with other usage models like modifying the devicetree at grub.

[johnstultz-work]

That's sort of the "pets vs cattle" difference though. On a desktop system you maintain, yes copying dtbo files over to the UEFI system partition and editing bootloader text files might make sense. But for many embedded devices, remotely flashing a provided partition via fastboot is much easier.

You mean, the user has to switch the board to fastboot mode everytime he wants to change the overlay? That doesn't sound like a nice UI.

Its not really a UI, but fastboot flashing is a very simple provisioning tool. Unplug the board, insert the mezzanine, boot to fastboot, flash the dtbo for the mezzanine and reboot. It can easily be done on a headless device without a serial console.

[robherring]

Note also that with EBBR as it is today the OS is already free to interfere with the hardware description if it wishes. For example an OS bootloader, such as grub, could implement a dtoverlay command to modify the device tree passed by the firmware before the kernel boots. A mechanism standardised using this approach would perhaps not be in scope for EBBR since it would be implemented purely within the OS.

I kind of assumed that EBBR's scope is stretched to all components till booting the kernel. Since EBBR targets standard booting procedure, providing a standard way to load dt overlays which in turn affecting the functionality made sense to me.

I think there are 2 parts to this we should keep separate. Where we store overlays and how we decide what overlays to apply.

First, I think we can agree there are overlays which should be stored alongside the base dtb(s). We've already defined that in the system partition. We may also have OS installed overlays. Their location too is defined at least to the extent we define where OS's put their files to avoid clobbering each other.

Second, there are multiple ways of deciding which overlays to apply. We'll need all of them. What's in scope for EBBR is the cases where users have to tell some component to load some overlay (e.g. I've plugged in non-discoverable mezzanine X). I don't think we want u-boot and Tianocore to each define their own thing here. Nor do we want to punt this to grub only and implicitly require grub or some other next stage bootloader if you want to use overlays in a consistent manner.

[agraf]

IIRC @ardbiesheuvel has already implemented manual and automatic overlay support in TianoCore for the Synquacer platform and 96boards mezzanines. Maybe all it takes is to push what he has done there into a common spec?

I do agree that we want to have this standardized across all implementations, but I'm also not 100% sure EBBR is the right place for it. I think this goes into a similar category as to where to put the DT GUID. Eventually we want to have a document that says "this is the GUID to pass a DT and these are the mechanisms users can expect to work with them". @glikely, any thoughts?

[glikely]

While I agree with the design goal, I don't think EBBR is the place to define it; or at least not yet.

EBBR is agnostic as to whether DT or ACPI is provided to the platform *providing firmware owns the DT/ACPI data. It doesn't care where that data is stored. It only cares that it gets provided to the OS, and the OS doesn't end up scribbling over it.

For firmware owns the overlays, then from the EBBR perspective, where & how DT overlays are stored is an internal detail. If firmware stores overlays in a partition, then the partition needs to be marked as owned-by-firmware so the OS doesn't scribble over them. If the OS (or OS preboot environment) needs to update overlays, then presumably update capsule is the right way to go about it.

However, if the OS owns the overlays, then the OS already has everything it needs for applying overlays before booting Linux. Overlays can be stored in whatever partition the OS chooses. The OS loader can get the DTB location out of the configuration table and can apply overlays before booting Linux.

In either case, EBBR doesn't specify the interface. I do think a common scheme should be designed and agreed upon, and the reference implementations (U-Boot & Tianocore) should implement them in the same way. Once that is done we can look if there are any parts of that which belong in EBBR.

[glikely]

I'm closing this issue, not because I think it is invalid, but because there is no action on EBBR at this point in time.