Do we separate /boot and the ESP

[ajeddeloh]

On Container Linux /boot and the ESP are the same. This is not the case on Fedora. Do we want to combine them for FCOS?

Pros of combining:

• Fewer partitions (simpler)
• Similar to CL for users migrating from CL

Cons of combining:

• /boot must be fat32
• Possibly confusing for users migrating from FAH

[dustymabe]

I honestly have never not had separate boot and ESP, but I like Fewer partitions (simpler).

[ajeddeloh]

I've honestly never not had combined boot and ESP =D

[lucab]

I have seen and used both, and I currently have preference for not having the ESP as /boot (but I can still be convinced otherwise). My concerns are:

• ESP usually has a small size and is easy to fill
• being fat32 precludes the usage of many Linux features (uid/gid, perms, xattrs, etc)
• (I think) it may preclude rpm-ostree from managing links/files under /boot
• in some environments (diskless PXE likely?) we may not have a ESP

But you do have a point on "fewer partitions". Especially in the "encrypted rootfs" scenario, I don't know how we could do that.

[cgwalters]

There was a big ~flamewar~ discussion about this on fedora-devel recently.

There were a lot of points brought up in that thread, but one of the most important is the BootloaderSpec Fedora Change which I really really want to do because currently with libostree we invoke grub2-mkconfig which invokes os-prober which is among the worst things in the base OS.

Here's a link to a comment I had in the thread.

[ajeddeloh]

wrt the Bootloader spec, I think it's fair to say that we don't care about dual booting. Automatic rollback almost makes things more complicated and I think it's perfectly okay to do things differently than fedora or other distros because of that. I just filed another issue about that. It has a proposal which would eliminate the need for ostree to even care about grub configs. So to some degree this issue is tied to how we do automatic rollbacks. I don't think following the systemd or fedora bootloader spec would actually buy us anything.

[mskarbek]

One thing to remember is that files under /boot are also labeled by SELinux. Combining /boot and ESP partitions will create a labeling problem and may upset a few people who will see SELinux warnings in their logs.

[dustymabe]

@mskarbek thanks for bringing that up and for discussing it during the meeting with us.

we discussed this at our meeting yesterday collectively our thoughts are:

• our goal is to make a combined boot and ESP partition, but we don't have super strong opinions here and foresee possible issues with fat32 not supporting xattrs. fallback plan is separate /boot and ESP

[dustymabe]

considering the SELinux limitation, is there any way we could move forward with ESP/boot combined?

[cgwalters]

There's not really any value to the SELinux labels in /boot.

[Conan-Kudo]

I personally do not think the loss of xattrs and other things is a good price to pay for unified smaller /boot because it's a FAT32 ESP.

[vtolstov]

what benefits of xattrs/selinux on /boot ? it contains mostly text files, sometimes initrd and kernel.

[dcode]

So, I could be mistaken, but an unlabelled /boot would mean that only unrestricted_t processes could modify it, unless runtime mount options gave it a specific type and policy associated with it. You would just need to ensure auto updates (locksmith? or just ostree?) would have the context to write to /boot. It's probably not a good practice to give a background periodic task unrestricted_t. Defeats the point of SELinux a bit.

[ajeddeloh]

Given the trouble with selinux/xattrs on fat32, and the fact we'd need to teach ostree to not try to write symlinks, I think we should separate them. /boot/efi will only contain the grub executable which will have it's prefix (where it looks for configs) set to /boot. This will allow us to keep as much as possible in /boot with only the bits needed in /boot/efi. I have a working PoC of this.