initramfs-tools script tries to mount legacy / none mountpoints

[xlotlu]

mountroot() calls mount_fs for all descendants of the root fs, which in turn does its best to mount everything, even special-casing "legacy" and "none" mountpoints, which should simply be skipped(?).

This happily drops the user to a shell with "filesystem cannot be mounted due to canonicalization error 2."

[FransUrbo]

It's kinda the point. Could you give me a (relevant if big) output from a zfs list so I know what you mean exactly?

[xlotlu]

Hi,

# zfs list -o name,mountpoint
NAME                                                                            MOUNTPOINT
abc                                                                             none
abc/ROOT                                                                        /
abc/ROOT/4139b767f5d66e1990d465e1f9c7afc9dc964cbe4df98f16c5cfbe2ba193b4d2       legacy
abc/ROOT/4139b767f5d66e1990d465e1f9c7afc9dc964cbe4df98f16c5cfbe2ba193b4d2-init  legacy
abc/ROOT/e6bfba268bf25406e8e07ec83926874a0a9ba0c789660d44e6b794f190888412       legacy
abc/home                                                                        /home
abc/storage                                                                     /storage
abc/var                                                                         /var
abc/var/cache                                                                   /var/cache

The ugly stuff under root was created because docker decided so, which is how I discovered the issue. But regardless of the purity of my setup, the issue I think is that it breaks expectations.

If the behaviour were to be in line with solaris, then legacy mountpoints should be handled during boot only if they showed up in fstab. While none should probably not be handled at all? If the user expressly marked the filesystem for non-mounting, this shouldn't be overruled.

Regards

[FransUrbo]

The ugly stuff under root was created because docker decided so, which is how I discovered the issue. But regardless of the purity of my setup, the issue I think is that it breaks expectations.

As far as I know, if put under something called "ROOT", it belongs to the OS and it will be very difficult to filter this out for one person, but still offer the functionality everyone else expects... If the behaviour were to be in line with solaris, then legacy mountpoints should be handled during boot only if they showed up in fstab.

This isn't Solaris. This is Linux, where we innovate and progress instead of standing still and doing what we've always done, wether it works or not :)

Just say'n! :).

While none should probably not be handled at all?

I'm fairly certain that it doesn't touch "none"s… Does it?

What version of packages are you using? And what dist?

[xlotlu]

Hey, sorry for the delay, extended afk.

As far as I know, if put under something called "ROOT", it belongs to the OS

I agree that polluting ROOT is ugly, but it's just a convention. I think the "official" convention of not auto-mounting legacy / none should prevail in this case.

(Off-topic: maybe I should report an issue with docker, but I don't quite know what to tell them... "Hey guys, this is ugly?" I'm sure from their point of view, they're doing the right thing.)

and it will be very difficult to filter this out for one person, but still offer the functionality everyone else expects...

True, special-casing is no good idea, but is this really functionality people expect?

I don't have the real-world experience to have an authoritative opinion, but as a techie, I'd expect to rely solely on the value of the mountpoint property: auto-mount things where I said they should, or don't mount them at all if I said they shouldn't. And let me do things the ugly way if I choose to.

instead of standing still and doing what we've always done, wether it works or not :)

... :D

I'm fairly certain that it doesn't touch "none"s… Does it?

I'm sure it does. I haven't tested it during boot, but this gets them recursively, and then they're treated similarly. The only difference is at line 344 - none would receive -o zfsutil, while legacy wouldn't, so I just tested this:

# zfs create -o mountpoint=none  abc/ROOT/woot
# zfs list -oname -tfilesystem -H -r abc/ROOT
abc/ROOT
abc/ROOT/4139b767f5d66e1990d465e1f9c7afc9dc964cbe4df98f16c5cfbe2ba193b4d2
abc/ROOT/4139b767f5d66e1990d465e1f9c7afc9dc964cbe4df98f16c5cfbe2ba193b4d2-init
abc/ROOT/e6bfba268bf25406e8e07ec83926874a0a9ba0c789660d44e6b794f190888412
abc/ROOT/woot
# mkdir /tmp/woot
# mount -o zfsutil -t zfs abc/ROOT/woot /tmp/woot
# mount | grep woot
abc/ROOT/woot on /tmp/woot type zfs (rw,relatime,xattr,noacl)

What version of packages are you using? And what dist?

0.6.5.5 on Debian testing.

[FransUrbo]

At first glance, this should be easy. Could you test this for me?

--- contrib/shell-common/zfs-functions.in       2016-03-31 19:37:44.265216681 +0200
+++ contrib/shell-common/zfs-functions.in.new   2016-03-31 19:40:30.341212685 +0200
@@ -416,6 +416,9 @@
                # above.
                [ "${f}" = "${fs}" ] && continue

+               # Skip filesystem with 'none' as mountpoint.
+               [ "$(get_fs_value "${fs}" mountpoint)" = "none" ] && continue
+
                zfs_action "Mounting filesystem ${f}" mount_fs "${f}" || \
                     return 1
        done

However, it might be very expensive if there's a lot of file systems below the root fs! But we deal with that if/when that happens. First we need to check if the patch works.

Apply it to your /usr/share/initramfs-tools/scripts/zfs file, rebuild your initrd (keep the old one save so you can use that to boot if this really don't work!!) and then reboot..

[FransUrbo]

Oups, missing ". And I managed to do the patch smaller. Check the comment (https://github.com/zfsonlinux/pkg-zfs/issues/196#issuecomment-204045652).

[xlotlu]

I'm not sure what your patch is against. My /usr/share/initramfs-tools/scripts/zfs looks exactly like pkg-zfs/contrib/initramfs/scripts/zfs.

In my case I solved it by replacing lines 324-350 with a return, which may or may not be correct depending on what you had in mind with the previous logic.

[FransUrbo]

I consider this fixed in 0.6.5.6-3. If not, please reopen and give additional information.

[xlotlu]

Nothing has changed. mountroot() gets all filesystems recursively: https://github.com/zfsonlinux/zfs/blob/zfs-0.6.5.7/contrib/initramfs/scripts/zfs#L931

and mount_fs() tries to forcefully mount filesystems with mountpoint = legacy / none: https://github.com/zfsonlinux/zfs/blob/zfs-0.6.5.7/contrib/initramfs/scripts/zfs#L323

which is incorrect behaviour.

Using 0.6.5.7, official debian packages. Reported downstream at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=826121

@FransUrbo please reopen this issue.

[tomtastic]

This bit me today too (also after a docker installation), system failed to boot with "rpool/ROOT/xxx cannot be mounted due to canonicalization error 2", referencing a docker volume with mountpoint set to legacy.

[travistabbal]

I got hit with this today, exact same reason. Installed Docker, which created those ZFS filesystems properly set with mountpoint=legacy. This script should not be auto-mounting legacy/none mountpoints!

I fixed it using the same patch method xlotlu mentioned. I used "return 0" in place of that code block and it works fine now.

"In my case I solved it by replacing lines 324-350 with a return, which may or may not be correct depending on what you had in mind with the previous logic."

[Asuranceturix]

I had the same problem. Other than patching the scripts, the simplest solution is to just create a dataset for /var/lib/docker prior to installing docker. Then, all those datasets that docker creates are children of this one and no one tries to mount them upon booting the machine:

rpool/docker                                                                   80.6M   163G  1.20M  /var/lib/docker
rpool/docker/065314f88d6eb0c3d71914adf014ea0ddb0518929dc2c7e0c4ed64d0f6c9adca  20.0M   163G  76.2M  legacy
rpool/docker/0b1d89ef85ca9cfa547fdb05a6301514bab26924fb05d1b8652ac98ed1d9cc7d   120K   163G  51.4M  legacy
rpool/docker/28c9680289a150dcf5ba45bb416e7ba631b38c78000416a723395bfd194cba7f   868K   163G  52.5M  legacy
rpool/docker/498172a5ff9ccb7bccbae8e8e8d1155ac7454a16b86ce9d3c134e23234519c5d  4.49M   163G  51.4M  legacy
rpool/docker/4d8f77533b75cc544b567add3c94c4b8c03b9a9b430151381e8fc4bea80c5c38  1.33M   163G  4.70M  legacy
rpool/docker/7ee0e235c2a7a4a66f2805519254d049142d4517c4bf5a732c4f5ad06cc2cea3  3.54M   163G  3.54M  legacy
rpool/docker/8ad1ecae764956f964d2a06e7e1e9ac0714cc63a9f5cd210f874eaad91acda17  5.23M   163G  57.1M  legacy
rpool/docker/9484fc6fb331edcbf4d04e1bfb5dc8093a848fd655eaae23220e7328e76117f3  43.4M   163G  47.6M  legacy
rpool/docker/e8173e407216a9ca76767caed347844f4f9c7d88538142249b6d90c22b3b71c5   440K   163G  51.7M  legacy
rpool/docker/fd005a35a002d6dd31ecff607fe385cb9d73a59f6bb0ad6ad7b2942f846e83cc   104K   163G  51.4M  legacy

[jbburns]

I ran into this today, also after installing docker. I wasn't able to find the script mentioned by @xlotlu. Setting the mountpoint to none had no effect, so instead I just destroyed all the docker datasets. Rebooted and was back to normal. wtf docker :)