`zfs mount` inside a chroot on a pool imported with -R can result in a double prefix

[ryao]

I had imported a pool with -R /mnt/gentoo. I then changed the mountpoint of the root dataset "rpool" from none to /rpool inside a chroot on /mnt/gentoo. This resulted in the dataset being mounted at /mnt/gentoo/mnt/gentoo/rpool while zfs list reported that it was mounted at /mnt/gentoo/rpool both outside and inside the chroot. Typically, zfs list inside a chroot will show the prefix, so that was to be expected. However, having the true mount location be /mnt/gentoo/mnt/gentoo/rpool was not.

mount inside the chroot will show rpool on /mnt/gentoo/rpool type zfs (rw,xattr) while mount outside the chroot will not show rpool at all. This can make it appear that a dataset is not mounted when it really is. It does not help that zfs umount rpool outside the chroot will claim that rpool is not mounted. On the bright side, /proc/mounts reports the correct location.

[rlaager]

I'm seeing this with at least Ubuntu 16.04 (if not 16.10). It prevents me from creating home directories properly inside the chroot.

[stale[bot]]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

[rlaager]

As far as I know (though I haven't retested recently), this is still occurring and it would be nice to fix for the OS install case (whether manual or automated).

[stale[bot]]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

[szubersk]

Guys, would you mind providing a minimal test case to showcase the alleged bug in here? I just run few tests and it seems to me that:

• there is no bug in there, mounts are visible from outside and inside of chroot
• double prefix is not a bug, this is a feature and a correct behavior. the new root is taken into the account when mounting a filesystem.

What creates a controversy in here is little bit counter-intuitive dichotomy of ZFS filesystem mountpoint props and actual VFS structure.

[szubersk]

System

/tmp/zfs % uname -a
Linux laptop.local 5.15.0-1-amd64 #1 SMP Debian 5.15.3-1 (2021-11-18) x86_64 GNU/Linux

/tmp/zfs % zfs --version
zfs-2.0.6-2+b1
zfs-kmod-2.1.1-1

Testcase

main() {
  chroot=$(pwd)/temp-chroot

  umount -Rl $chroot/* &>/dev/null ||:
  umount -Rl $chroot &>/dev/null ||:
  zpool export -f testpool &>/dev/null ||:
  mkdir -p $chroot

  zpool import -d $(pwd)/disk.img -N testpool
  zfs set mountpoint=none testpool
  zpool export -f testpool
  zpool import -d $(pwd)/disk.img -R $chroot testpool

  for d in /dev /proc /sys; do
    mount --rbind --make-rprivate $d "$chroot/$d"
  done

  echo INSIDE OF CHROOT
  chroot $chroot bash -c "set -e ; zfs set mountpoint=/testpool testpool ; zpool list testpool ; echo ; zfs list -r testpool ; echo ; mount | grep testpool"

  echo
  echo OUTSIDE OF CHROOT
  zpool list testpool
  echo
  zfs list -r testpool
  echo
  mount | grep testpool
}

main "$@"

Results

/tmp/zfs % doas ~/projects/test.sh
INSIDE OF CHROOT
NAME       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
testpool   288M   219K   288M        -         -     0%     0%  1.00x    ONLINE  /tmp/zfs/temp-chroot

NAME       USED  AVAIL     REFER  MOUNTPOINT
testpool   176K   160M       24K  /tmp/zfs/temp-chroot/testpool

/testpool on /tmp/zfs/temp-chroot/testpool type zfs (rw,xattr,noacl)

OUTSIDE OF CHROOT
NAME       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
testpool   288M   219K   288M        -         -     0%     0%  1.00x    ONLINE  /tmp/zfs/temp-chroot

NAME       USED  AVAIL     REFER  MOUNTPOINT
testpool   176K   160M       24K  /tmp/zfs/temp-chroot/testpool

testpool on /tmp/zfs/temp-chroot/tmp/zfs/temp-chroot/testpool type zfs (rw,xattr,noacl)

Conclusions Everything seems OK to me: altroot property of the pool is unchanged by mount and chroot operations, mountpoint property of the dataset is adjusted correctly by altroot value. The thing, in my opinion correct but could be contested in this issue, is the values returned by mount command. They make perfect sense, when we try to compute the final mountpoint: take the dataset.mountpoint, adjust by zpool.altroot and, finally, adjust by the chroot.

The intuitive behavior expectation would suggest that the dataset.mountpoint should also be adjusted by chroot (as well as pivot_root on Linux and reboot -r on FreeBSD, possibly other VFS manipulations). I am not sure if OpenZFS, in current form, would support that kind of "viewpathing".

Workaround / Circumvention To work this behavior around (e. g. in initrd) one of following sequences are used

zpool import -N testpool
chroot /tmp/zfs/temp-chroot
# now `zfs mount` whatever you want

or

zpool import -N testpool
mount -t zfs -o zfsutil testpool /tmp/zfs/temp-chroot
chroot /tmp/zfs/temp-chroot
zfs mount -a

as mount -t zfs -o zfsutil doesn't change the zpool.altroot nor dataset.mountpoint properties.