search

trapexit / mergerfs

a featureful union filesystem
http://spawn.link
Other
4.18k stars 170 forks source link

`pivot_root` to a mergerfs union mount just hangs #935

Open eadwu opened 3 years ago

eadwu commented 3 years ago

Describe the bug

Something I found out messing around with namespaces and chroots, didn't see anything in the issues with regards to pivot_root so putting it here. It hangs and could only be killed through direct signals kill -9 $PID to the parent unshare since Ctrl+C doesn't work.

To Reproduce

Warning: Leaves hanging processes ... haven't figured out to kill the processes when unshare is killed, had to manually find each running process (thankfully not a lot mergerfs and pivot_root mostly)

cd "$(mktemp -d)"
unshare --pid --mount --map-root-user --fork --mount-proc --propagation private
mkdir -p chroot
mergerfs -o use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs -o auto_unmount /=RW chroot

cd chroot
mount -t proc none proc/
mount -o rbind /dev dev/
mount -t devpts none dev/pts

pivot_root . . # or pivot_root . old_root but requires an accessible directory /old_root

Expected behavior

pivot_root to finish executing cleanly.

System information:

Additional context

overlayfs can pivot_root correctly with lowerdir of type ZFS and upperdir and workdir of type tmpfs.

fuse-overlayfs can pivot_root correctly.

unionfs-fuse supposedly can pivot_root correctly, apparently it needs some -o chroot workaround to prevent a deadlock which may be the problem here as well.

trapexit commented 3 years ago

No, it doesn't handle loops of itself which appears to be what happens. I'm not sure what the expected behavior is. I'd have to look at the other projects.

trapexit commented 3 years ago

I'm not sure.

fuse-overlayfs does not have anything specific wrt pivot_root or chroot. unionfs-fuse calls chroot but when doing what you show above it just blocks. I've not tried unionfs-fuse with chroot option following what you did above to see if it too has a similar behavior.

If the expectation is that mergerfs is going to hold open file descriptors to the original branch paths that's not happening. Not now anyway. That would substantially change the way mergerfs works. It would not only impact the way it works but would change the user experience with regard to adding/removing branches, failure conditions, etc.

eadwu commented 3 years ago

I'm not really sure what pivot_root expects so I'm not sure about the expectations. Though it's fine on my end since pivot_root looks like it's just used to prevent escaping to the original root within the namespace which doesn't really matter since I expose the original root (so chroot works just fine).

I included the reproduction code for unionfs-fuse and fuse-overlayfs in which pivot_root works in case it's needed in the future.

For unionfs-fuse

unshare --pid --mount --map-root-user --fork --mount-proc --propagation private

CHROOT_PATH="$(mktemp -d)"
mkdir -p "${CHROOT_PATH}/root"
mount -o rbind / "${CHROOT_PATH}/root"

UNION_ROOT="$(mktemp -d)"
unionfs -o allow_other,use_ino -o cow,chroot="${CHROOT_PATH}" /root=RW "${UNION_ROOT}"

cd "${UNION_ROOT}"
mount -t proc none proc/
mount -o rbind /dev dev/
pivot_root . .

and for fuse-overlayfs

unshare --pid --mount --map-root-user --fork --mount-proc --propagation private

WORK_DIR="$(mktemp -d)"
mkdir -p "${WORK_DIR}"/{lowerdir,workdir,union}

mount -t tmpfs none "${WORK_DIR}/lowerdir"
mount -t tmpfs none "${WORK_DIR}/workdir"
fuse-overlayfs -olowerdir="${WORK_DIR}/lowerdir",upperdir=/,workdir="${WORK_DIR}/workdir" "${WORK_DIR}/union"

cd "${WORK_DIR}/union"
mount -t proc none proc/
mount -o rbind /dev dev/
pivot_root . .
trapexit commented 3 years ago

I think I got it working with a chroot similar to unionfs-fuse though I need to fool with it more.

That are you expecting to run this as non-root? Because that's not really a supported usecase by mergerfs.

eadwu commented 3 years ago

I'll personally be running as root most of the time if not all with just chroot. I was just messing around with unshare since other people might be interested but can only run it unprivileged.