openzfs / openzfs-docs

OpenZFS Documentation
https://openzfs.github.io/openzfs-docs/
135 stars 194 forks source link

Question: How are GRUB updates handled with mirror topology on ZFS root? #506

Closed lonkaars closed 5 months ago

lonkaars commented 6 months ago

I followed the Debian Bookworm Root on ZFS instructions, and they worked flawlessly. I am using UEFI boot with a mirror topology on the boot drive (or its partitions?). This is my first time using ZFS, so apologies if these are stupid questions.

I installed GRUB as described in step 5, and cloned the efivars(?) partition using dd as described in step 6.7. As I currently understand it, the install instructions end up with /boot and /boot/EFI on separate partitions, and only the /boot partition is handled as the bpool pool under ZFS (and thus gets mirrored automatically). I believe GRUB is automatically updated after each kernel upgrade, but I am currently unsure if these updates are handled completely automatically, or if the dd or grub-install steps should be manually repeated after each kernel upgrade.

I am unsure about the following:

gmelikov commented 6 months ago

Not sure if I remember everything correctly, but the easiest way to mirror your EFI partition is to just have individual EFI partitions on each of your disks, and bootstrap them with only minimal (for example) GRUB loader, which will load initramfs from BOOT zfs pool afterwards.

Don't want to lie here, so it should be checked if Debian nowadays has any mechanisms to bootstrap all EFI partitions on their existence. It's a good topic to describe additionally in our docs, though.

eduarddejong commented 5 months ago

Personally I'm not running my Linux system root itself on ZFS (I also haven't dived into how to do that either), and mounting may work differently then, but I believe I may still have your answer.

Generally as soon as the ESP has been properly mounted when your system has been booted, it should all work automatically during updates. ZFS, or any other file system used for your system root, should not change that software behaviour.

The grub-install utility, when you don't give it a destination directory manually, looks for /boot/efi by default, which is then supposed to be the ESP mount point, and it should be defined in your /etc/fstab file so that it's always mounted.

I have quite a lot of personal experience with tweaking GRUB installations for multiple Linux distributions.

For example I know on Arch Linux the package manager never automatically updates the actual GRUB loader itself, but just the GRUB package on the system when a new version is available. And I am actually happy with that behaviour too. I just update my GRUB loader manually when I want on Arch, especially because I have some specific customizations there and I use my own written script for that.

But distros like Debian or Fedora also automate that process more, they do indeed update the actual loader when a new GRUB version is available. Still the underlying grub update goes the same, when any ESP on '/boot/efi' has been correctly mounted, it will al work without worries.

But it goes further. To prevent conflicts between grub updates from multiple distros in my dual booted setups that I often make, I often just make a separate ESP/FAT partition for each specific installed system (and define that one in it's own fstab). It can all literally sit on the same drive as well if you just do it right. No need to buy more drives, even though people often believe that. (I just maintain extra menu options to chainload the other GRUB loaders from the initial one, so that I can choose any system, and I also do that for a Windows boot loader too).