NVIDIA / pyxis

Container plugin for Slurm Workload Manager
Apache License 2.0
263 stars 28 forks source link

--container-mounts can't mount directory with spaces #133

Open mathrock74 opened 5 months ago

mathrock74 commented 5 months ago

Mounting a directory with spaces to the container doesn't work:

~$ srun --no-container-mount-home --container-mounts '/muw/home/my_user/foo bar':/root --container-image=debian date pyxis: importing docker image: debian pyxis: imported docker image: debian slurmstepd: error: pyxis: container start failed with error code: 1 slurmstepd: error: pyxis: printing enroot log file: slurmstepd: error: pyxis: enroot-mount: failed to mount: /muw/home/my_user/foo at /tmp/enroot-data/user-36209/pyxis_479.0/bar: No such file or directory slurmstepd: error: pyxis: couldn't start container slurmstepd: error: spank: required plugin spank_pyxis.so: task_init() failed with rc=-1 slurmstepd: error: Failed to invoke spank plugin stack srun: error: deepops-node1: task 0: Exited with exit code 1

I don't know if this is pyxis or enroot issue.

pyxis: 0.16.1 enroot: 3.4.1 pyxis options: container_scope=global

Thanks for help.

flx42 commented 5 months ago

This might require a few tweaks in pyxis for this use case, for sure.

On the enroot side, I'm not sure if it will require any change or not. I tried quoting the path with a space but without success:

$ file '/home/fabecassis/foo bar'
/home/fabecassis/foo bar: directory

$ cat conf
mounts() {
        echo '"/home/fabecassis/foo bar" "/mnt" x-create=auto,rbind'
}

$ enroot start --conf ./conf ubuntu
enroot-mount: failed to mount: "/home/fabecassis/foo at /tmp/enroot-data/user-1000/ubuntu/bar": No such file or directory

@3XX0 can you please check if this use case is supported in enroot today?

3XX0 commented 4 months ago

This is doable with enroot built against glibc but not musl unfortunately (which is shipped by default).

mounts() {
        echo 'foo\040bar /mnt x-create=auto,rbind'
}
flx42 commented 4 months ago

Ah yes that's unfortunate.

But it seems that when enroot is compiled with glibc, pyxis will simply work with the \040 syntax as it's just being passed through to enroot:

$ srun --container-image=ubuntu:22.04 --container-mounts '/home/fabecassis/foo\040bar:/mnt' findmnt /mnt
TARGET SOURCE                                   FSTYPE OPTIONS
/mnt   /dev/nvme0n1p2[/home/fabecassis/foo bar] ext4   rw,relatime,errors=remount-ro