mineshaftgap / d4m-nfs

Docker for Mac with NFS for performance improvements over osxfs
296 stars 26 forks source link

Mounting directories that do not exist on the host? #25

Closed alanfranz closed 7 years ago

alanfranz commented 7 years ago

Hello, I often run into what I think it's a quirk of d4m-nfs: if the host directory (or file, I suppose) does not exist, the volume mount fails. osxfs and native Linux docker work properly in such scenario, by just creating the host directory.

This works 100% (tmpfs is on osxfs):

docker run -v /tmp/something-nonexisting:/sth -it centos:7 /bin/bash

this fails even though /Users is shared:

docker run -v /Users/alan/something-nonexisting:/sth -it centos:7 /bin/bash

docker: Error response from daemon: Mounts denied: fo.
.

The path /Users/alan/something-nonexisting
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more in.

this works 100%:

mkdir -p /Users/alan/something-nonexisting
sleep 10
docker run -v /Users/alan/something-nonexisting:/sth -it centos:7 /bin/bash

(additionally, there seems to be a sort of 'race condition', the new nfs dir on the host is not recognized immediately inside the docker vm).

Any idea of how the issue could be fixed/worked around?

if-kenn commented 7 years ago

@alanfranz if you want it to automatically create the directory you will need to give it access to do that in d4m-nfs/etc/d4m-nfs-mounts.txt (using root/wheel) like the following:

/Users/alan:/Users/alan:0:0

This works for me with the above configuration: docker run -v /Users/kenn/foohjdfjfdjfd:/fjjf -it alpine /bin/sh

if-kenn commented 7 years ago

No response from issue author within 2 weeks, closing.

alanfranz commented 7 years ago

Hello @if-kenn , thanks for your response, I'm sorry it took a bit for me to check.

Your proposed solution doesn't work for me:

AlanMacbook:d4m-nfs alan$ docker run -v /Users/alan/asdasdasd:/asdasdasd -it ubuntu:xenial /bin/bash
docker: Error response from daemon: Mounts denied:
The path /Users/alan/asdasdasd
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
..

(of course everything works 100% if I choose an existing host directory)

my d4m-nfs-mounts.txt:

/Users:/Users:0:0
/Volumes:/Volumes
/private:/private

/etc/exports:

# d4m-nfs exports

"/Users" -alldirs -mapall=0:0 localhost
"/Volumes" -alldirs -mapall=0:0 localhost
"/private" -alldirs -mapall=0:0 localhost

nfs mounts in d4m machine:

/ # cat /proc/mounts | grep nfs
192.168.65.1:/Users /Users nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.65.1,mountvers=3,mountproto=tcp,local_lock=all,addr=192.168.65.1 0 0
192.168.65.1:/Volumes /Volumes nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.65.1,mountvers=3,mountproto=tcp,local_lock=all,addr=192.168.65.1 0 0
192.168.65.1:/private /private nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.65.1,mountvers=3,mountproto=tcp,local_lock=all,addr=192.168.65.1 0 0
alanfranz commented 7 years ago

I'm using Docker for Mac Version 1.13.0 (15072) on Sierra, and d4m-nfs @ 559d55db894a15a6a378f3d013862e867559e00d

alanfranz commented 7 years ago

image

alanfranz commented 7 years ago
AlanMacbook:d4m-nfs alan$ cat /tmp/d4m-mount-nfs.sh
ln -nsf /tmp/d4m-apk-cache /etc/apk/cache
apk update
apk add nfs-utils sntpc
rpcbind -s > /dev/null 2>&1

DEFGW=$(ip route|awk '/default/{print $3}')
FSTAB="\n\n# d4m-nfs mounts\n"

if false && ! $(grep ':/mnt' /tmp/d4m-nfs-mounts.txt > /dev/null 2>&1); then
  mkdir -p /mnt

  FSTAB="${FSTAB}${DEFGW}:/Users/root /mnt nfs nolock,local_lock=all 0 0"
fi

if [ -e /tmp/d4m-nfs-mounts.txt ]; then
  while read MOUNT; do
    DSTDIR=$(echo "$MOUNT" | cut -d: -f2)
    mkdir -p ${DSTDIR}
    FSTAB="${FSTAB}\n${DEFGW}:$(echo "$MOUNT" | cut -d: -f1) ${DSTDIR} nfs nolock,local_lock=all 0 0"
  done < /tmp/d4m-nfs-mounts.txt
fi

if ! $(grep "d4m-nfs mounts" /etc/fstab > /dev/null 2>&1); then
    echo adding d4m nfs config to /etc/fstab:
    echo -e $FSTAB | tee /etc/fstab
else
    echo d4m nfs mounts already exist in /etc/fstab
fi

sntpc -i 10 ${DEFGW} &

sleep .5
mount -a
touch /tmp/d4m-done
AlanMacbook:d4m-nfs alan$ cat /tmp/d4m-nfs-mounts.txt
/Users:/Users:0:0
/Volumes:/Volumes
/private:/private