dasJ / sd-zfs

Compatibility between systemd and ZFS roots
MIT License
43 stars 13 forks source link

Use of mountpoint or org.zol:mountpoint #27

Open johnlane opened 6 years ago

johnlane commented 6 years ago

I am using ArchLinux on a ZFS root file system, booting with systemd and sd-zfs (mkinitcpio-sd-zfs). I am experimenting with multiple root filesysem datasets, either cloned from snapshots or created from scratch. I am unsure how to specify their mount points because they are all the same (/).

The README here says

The mountpoint value of the dataset needs to be /.

Setting the mountpoint property to the same value on multple datasets seems to cause more problems than it solves.

I noticed that snapshot clones created by sd-zfs do not have the mountpoint property but they work fine. My own datasets don't.

I've done tests and know that the initramfs won't mount the root filesystem without a defined mount point. So I dug further and found another property called org.zol:mountpoint which those snapshot clones do have. I tried setting that on my own datasets and it works.

I've read over the source for sd-zfs and it sets this property when cloning and it uses it when mounting.

So, is this some undocumented feature or is it the right way to do it - I don't know. Googling hasn't helped with this.

Booting a dataset either explicitly as tank/ROOT/arch123 or with AUTO and the bootfs property works fine with this org.zol.mountpoint. In fact, they work without setting mountpoint at all. It also works on other distros as well as Arch.

You can set it easily with set org.zol.mountpount=/ tank/ROOT/arch123 and, unlike setting mountpoint, it doesn't cause ZFS to try mounting (or unmounting) something you don't want or need (and avoids the associated error messages).

SO it works nicely but isn't documented anywhere that I could find. So I ask, is it ok to do it this way, or is there a better way?

If this is the correct approach then perhaps the README could be updated to reflect this.

guyest commented 5 years ago

I haven't investigated this extensively, but "it doesn't smell right". I also run arch on ZFS (except /boot) with the sd-zfs hook and none of my datasets have the org.zol:mountpoint property defined. My setup is a bit different because I have several disks which are all encrypted so they are unlocked before any pools can be created from them, but the basic story is the same. I set up my datasets in a similar way to you with the intent of leaving myself the option of having multiple boot environments. Though I have not yet put that to the test, I had envisioned that this would all be handled by setting the bootfs property and using the zfs:AUTO kernel option. If I want a different boot environment, just change the flag --- done.

I get that you're trying something a bit different since you want to use a snapshot explicitly and sd-zfs is supposed to handle that by replicating that snapshot into a completely new dataset and then booting from that new dataset. I would think the most straightforward way of handling that is simply to unset the bootfs property on whatever dataset of that pool which previously had it and then set the bootfs property on the newly created dataset (replicated from the snapshot). Is that not what you're finding actually takes place? (Looking at the code it doesn't seem like it, so I would guess that is what you find but I could have missed it.)

In any event, since booting from a snapshot is not a supported feature of of ZOL it is definitely possible that a subtle oversight was made. This is something I would like to have handled cleanly as well so I will try to replicate your scenario on one of my machines when I get the chance.

mbrgm commented 4 years ago

@guygma Any news on this?

ryao commented 1 year ago

The colon means that it is a user property. Some daemons use user properties for adding metadata to ZFS datasets for their use. Interestingly, this entered the repository via the contrib directory, but it is not an official thing. If it were official, it would not have a colon.

johnlane commented 1 year ago

Just wanted to say that I'm still relying on this so I can boot a new root filesystem without affecting the old one. Just to be clear, I am booting datasets, not snapshots. I create a new dataset mounted to a temporary location so I can copy a roofts into it that was preapred elsewhere. I unmount (clearing the mountpoint) and set the aforementioned org.zol:mountpoint to / and I can then boot any such dataset from grub without having to fiddle with zfs attributes such as the bootfs property. I usually keep 3 or 4 such datasets around. I've been working like this for a few years now without any problems.

samarium commented 1 year ago

I've just been digging thru the Ubuntu /usr/share/initramfs-tools/scripts/zfs and came across org.zol:mountpoint in my quest to better understand boot environments, especially with sub datasets, since just a root file systems seems to be usable just fine. I've wondering you you have any datasets under the root dataset, or only the root dataset which you are booting. If you have sub datasets, then I'll be interested in hearing how that went and what you had to do, otherwise I'll just have to experiment myself. I've also been looking at the arch/endeavouros dracut implementation, but haven't got to the arch/manjaro implementation as yet, which is mkinitcpio based IIRC.

I would be interested in seeing the zfs get output from some of your environments if possible please:

zfs get -r -t filesystem canmount,mountpoint,org.zol:mountpoint pool/.../root-active pool/.../root-inactive | sort -b -k2,2 -k1,1

Thanks, and I can let you know how I go too, if that is interesting to you.

johnlane commented 1 year ago

In the most basic example (see my updated gist) there is only one dataset but in my actual "live" systems there are multiple - I create new ones when I install an upgraded rootfs and I keep a few old ones, so I may have 4 or 5 datasets. There are no child datasets beneath those roots. Here's an example of my laptop right now where there are 2 datasets:

NAME                                                                                          USED  AVAIL     REFER  MOUNTPOINT
system                                                                                        919G  3.76G       96K  none
system/archlinux                                                                             85.0G  3.76G       96K  none
system/archlinux/desktop-20220216                                                            54.4G  3.76G     43.5G  none
system/archlinux/desktop-20220725                                                            30.7G  3.76G     28.0G  none

and

$ zfs get -r -t filesystem canmount,mountpoint,org.zol:mountpoint system/archlinux/desktop-20220216 system/archlinux/desktop-20220725 | sort -b -k2,2 -k1,1
system/archlinux/desktop-20220216  canmount            on                  default
system/archlinux/desktop-20220725  canmount            on                  default
system/archlinux/desktop-20220216  mountpoint          none                local
system/archlinux/desktop-20220725  mountpoint          none                local
system/archlinux/desktop-20220216  org.zol:mountpoint  /                   local
system/archlinux/desktop-20220725  org.zol:mountpoint  /                   local

I found problems if more than one dataset had mountpoint set to the same mountpoint (such as /). I have mountpoint set to none and I have instead set org.zol:mountpoint to / (it's fine to set this to the same value on multiple datasets).

In the simple example in the gist where there is only one dataset, it is ok to set mountpoint and not set org.zol:mountpoint. I've thrown together an example that creates two rootfs, it does not work if mountpoint is used but does work if org.zol:mountpoint is used.