search

EugenMayer / docker-sync

Run your application at full speed while syncing your code for development, finally empowering you to utilize docker for development under OSX/Windows/*Linux
GNU General Public License v3.0
3.53k stars 289 forks source link

windows 10 fails to mount volume #673

Closed lmapii closed 5 years ago

lmapii commented 5 years ago

Error/Feature Requestion/Docs

docker is not able to start the container even though the volume exists. full dump:

user@pc6782:/c/_vhd/make-ttc-2000-flat$ docker-sync-stack start
/var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/update_check.rb:41: warning: Insecure world writable dir /home/user/.local/bin in PATH, mode 040777
       note:  You can also run docker-sync in the background with docker-sync start
     command  docker volume create --opt type=none --opt device="/c/_vhd/make-ttc-2000-flat/" --opt o=bind --name builder-sync
     success  Docker volume for builder-sync created
     success  Watcher disabled by configuration
          ok  Found implicit docker-compose-dev.yml and using it from /c/_vhd/make-ttc-2000-flat/docker-compose-dev.yml
          ok  starting compose
Creating network "make-ttc-2000-flat_default" with the default driver
Creating make-ttc-2000-flat_builder_1 ... error

ERROR: for make-ttc-2000-flat_builder_1  Cannot start service builder: error while mounting volume '/var/lib/docker/volumes/builder-sync/_data': failed to mount local volume: mount /c/_vhd/make-ttc-2000-flat/:/var/lib/docker/volumes/builder-sync/_data, flags: 0x1000: no such file or directory

ERROR: for builder  Cannot start service builder: error while mounting volume '/var/lib/docker/volumes/builder-sync/_data': failed to mount local volume: mount /c/_vhd/make-ttc-2000-flat/:/var/lib/docker/volumes/builder-sync/_data, flags: 0x1000: no such file or directory
Encountered errors while bringing up the project.
#<Thread:0x00007fffec389d40@/var/lib/gems/2.5.0/gems/docker-sync-0.5.11/tasks/stack/stack.thor:50 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
        3: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/tasks/stack/stack.thor:51:in `block in start'
        2: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/compose.rb:42:in `run'
        1: from /var/lib/gems/2.5.0/gems/docker-compose-1.1.12/lib/docker/compose/session.rb:98:in `up'
/var/lib/gems/2.5.0/gems/docker-compose-1.1.12/lib/docker/compose/session.rb:270:in `run!': 'up' failed with status 'pid 10846 exit 1': Creating network "make-ttc-2000-flat_default" with the default driver (Docker::Compose::Error)
EXCEPTION: #<Docker::Compose::Error: 'up' failed with status 'pid 10846 exit 1': Creating network "make-ttc-2000-flat_default" with the default driver>
MESSAGE: 'up' failed with status 'pid 10846 exit 1': Creating network "make-ttc-2000-flat_default" with the default driver

Docker Driver

Docker version 19.03.1, build 74b1e89 docker-compose version 1.24.1, build 4667896

Notice: I haven't linked install .exe files within the WSL but have installed docker and docker-compose in there, pointing at the docker instance on windows. (export DOCKER_HOST=tcp://localhost:2375)

Sync strategy

default

Files

docker-sync.yml

version: "2"
options:
  verbose: true
syncs:
  builder-sync:
    src: '.'
    sync_excludes: ['.vscode', '.git']

docker-compose.yml

version: "3.4"
services:
  builder:
    image: tricore-builder

docker-compose-dev.yml

version: "3.4"
services:
  builder:
    volumes:
      - builder-sync:/project:nocopy
    entrypoint: tail -f /dev/null

volumes:
  builder-sync:
    external: true

OS

Windows Version Enterprise 10.0.17134 Build 17134

Info

The volume seems to have been created by docker-sync/compose:

user@pc6782:/c/_vhd/make-ttc-2000-flat$ docker volume ls
DRIVER              VOLUME NAME
local               be1aaf776e67c1022977d7391aab905db53047102023224f23fa2c622d13a236
local               builder-sync
user@pc6782:/c/_vhd/make-ttc-2000-flat$
user@pc6782:/c/_vhd/make-ttc-2000-flat$ docker volume inspect builder-sync
[
    {
        "CreatedAt": "2019-08-23T14:01:31Z",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/builder-sync/_data",
        "Name": "builder-sync",
        "Options": {
            "device": "/c/_vhd/make-ttc-2000-flat",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]
user@pc6782:/c/_vhd/make-ttc-2000-flat$

I was trying to use docker-sync to circumvent the famous Windows 10 problem where mounting drives is not possible due to firewall rules (in my case domain rules). I don't know the internals of docker-sync and was hoping it would keep the volume up to sync without this setting.

Is my setup failing due to shared drives not being available?

MustafaAlshehab commented 5 years ago

I think I had this issue before. Try starting up docker-sync with sudo and specify the sync strategy and sync_host_ip on docker-sync.yml ` version: "2" options: verbose: true syncs: builder-sync: sync-strategy: 'unison' src: './' sync_excludes: ['.vscode', '.git'] sync_host_ip: '127.0.0.1' #host ip isn't properly inferred sync_args: ['-perms=0'] #required for two way sync ie generators, etc '

lmapii commented 5 years ago

Hi @MustafaAlshehab , thanks a lot for the feedback. I included your changes and it seems to be working. What I did to get it up and running:

docker-compose was not installed for sudo, thus I had to:

sudo ln -s /home/user/.local/bin/docker-compose /usr/bin/docker-compose

Also the .bashrc from my user doesn't load for sudo thus I have to prepend DOCKER_HOST=tcp://localhost:2375 to all my commands, a simple makefile helps:

stack-start:
    sudo DOCKER_HOST=tcp://localhost:2375 docker-sync-stack start

stack-clean:
    sudo DOCKER_HOST=tcp://localhost:2375 docker-sync-stack clean

I've also included the changes to the sync-file:

version: "2"

options:
  verbose: true
syncs:
  builder-sync:
    src: './'
    sync_strategy: 'unison'
    sync_excludes: ['.vscode', '.git']
    sync_host_ip: '127.0.0.1'
    sync_args: ['-perms=0']

It is interesting that docker-sync-stack needs to be called with sudo since any other docker commands didn't have to do that. If I run it without sudo unison doesn't seem to start properly:

user@pc6782:/c/_vhd/make-ttc-2000-flat$ docker-sync-stack start
/var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/update_check.rb:41: warning: Insecure world writable dir /home/user/.local/bin in PATH, mode 040777
       note:  You can also run docker-sync in the background with docker-sync start
          ok  Starting unison for sync builder-sync
          ok  builder-sync container not running
          ok  Starting precopy
     precopy  docker run --rm -v "builder-sync:/app_sync" -e APP_VOLUME=/app_sync -e TZ=$(basename $(dirname `readlink /etc/localtime`))/$(basename `readlink /etc/localtime`) -e UNISON_SRC="-socket 5000" -e UNISON_DEST="/app_sync" -e MONIT_ENABLE="false" -e MONIT_INTERVAL="" -e MONIT_HIGH_CPU_CYCLES="" -e UNISON_ARGS="-ignore='Name .vscode' -ignore='Name .git'" -e UNISON_WATCH_ARGS="" -e HOSTSYNC_ENABLE="0" -e UNISONSOCKET_ENABLE="1"  --name builder-sync eugenmayer/unison:2.51.2.1 /usr/local/bin/precopy_appsync
dirname: missing operand
Try 'dirname --help' for more information.
basename: missing operand
Try 'basename --help' for more information.
basename: missing operand
Try 'basename --help' for more information.
doing initial sync with unison
Unison 2.51.2 (ocaml 4.06.1): Contacting server...
Looking for changes
Reconciling changes
Propagating updates
UNISON 2.51.2 (OCAML 4.06.1) started propagating changes at 07:32:36.48 on 06 Sep 2019
[BGN] Copying  from /app_sync to /host_sync
[END] Copying
UNISON 2.51.2 (OCAML 4.06.1) finished propagating changes at 07:32:36.48 on 06 Sep 2019
Saving synchronizer state
Synchronization complete at 07:32:36  (1 item transferred, 0 skipped, 0 failed)
real    0m 0.08s
user    0m 0.05s
sys     0m 0.01s
chown ing file to uid 0
          ok  creating builder-sync container
     command  docker run -p '127.0.0.1::5000' -v builder-sync:/app_sync -e APP_VOLUME=/app_sync -e TZ=$(basename $(dirname `readlink /etc/localtime`))/$(basename `readlink /etc/localtime`) -e UNISON_SRC="-socket 5000" -e UNISON_DEST="/app_sync" -e MONIT_ENABLE="false" -e MONIT_INTERVAL="" -e MONIT_HIGH_CPU_CYCLES="" -e UNISON_ARGS="-ignore='Name .vscode' -ignore='Name .git'" -e UNISON_WATCH_ARGS="" -e HOSTSYNC_ENABLE="0" -e UNISONSOCKET_ENABLE="1"  --name builder-sync -d eugenmayer/unison:2.51.2.1
dirname: missing operand
Try 'dirname --help' for more information.
basename: missing operand
Try 'basename --help' for more information.
basename: missing operand
Try 'basename --help' for more information.
          ok  starting initial sync of builder-sync
     command  docker inspect --format='{{(index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort}}' builder-sync
     command  unison -testserver /app_sync "socket://127.0.0.1:32771"
Traceback (most recent call last):
        15: from /usr/local/bin/docker-sync-stack:23:in `<main>'
        14: from /usr/local/bin/docker-sync-stack:23:in `load'
        13: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/bin/docker-sync-stack:15:in `<top (required)>'
        12: from /var/lib/gems/2.5.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
        11: from /var/lib/gems/2.5.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
        10: from /var/lib/gems/2.5.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
         9: from /var/lib/gems/2.5.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
         8: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/tasks/stack/stack.thor:46:in `start'
         7: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_manager.rb:110:in `run'
         6: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_manager.rb:110:in `each'
         5: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_manager.rb:111:in `block in run'
         4: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_process.rb:85:in `run'
         3: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_strategy/unison.rb:39:in `run'
         2: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_strategy/unison.rb:211:in `start_container'
         1: from /var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_strategy/unison.rb:211:in `loop'
/var/lib/gems/2.5.0/gems/docker-sync-0.5.11/lib/docker-sync/sync_strategy/unison.rb:216:in `block in start_container': Failed to start unison container in time, try to increase max_attempt (currently 5) in your configuration. See https://github.com/EugenMayer/docker-sync/wiki/2.-Configuration for more informations (RuntimeError)
EugenMayer commented 5 years ago

docker-sync will not need sudo at all, it should never need that.

You have a broken setup, either your gems leaked into the system, instead of a scoped ruby for the user or something else leaked into the system space, now failing if you do not use sudo ( which will fail itself )

lmapii commented 5 years ago

OK the sudo issue might have been an artifact of when I tried to sudo docker-sync-stack start. The files in the home directory still existed, simply removing .unison and unison.log resolved that.

In summary: Adding sync_strategy, sync_host_ip and sync_args fixed the issue for me, less arguments may be actually required. Thanks for the help!

EDIT: Anyone running into similar permission/problems might simply want to check if unison and unison-fsmonitor can be executed as user. This one was the indication on my case:

user@pc6782:/some/silly/path$ unison-fsmonitor
VERSION 1
user@pc6782:/some/silly/path$ unison
Error: Could not write preferences file (/home/user/.unison/default.prf: Permission denied)
EugenMayer commented 5 years ago

Tahnks for reporting back @lmapii !