User is always mapped to root with unison

[ShurikAg]

I am trying to use docker-sync for my project, and everything seems to be working well except permissions. Maybe it is not an issue, but I simply don't understand how to use it.

In my Dockerfile I am forcing www-data to be with a specific user id:

FROM php:7-fpm
...
RUN usermod -u 5000 www-data
...

In docker-sync.yml I set:

...
sync_strategy: 'unison'
sync_user: 'www-data'
sync_userid: '5000'
...

When the container starts, all synced files are set to root:www-data.

I was expecting to have it set to www-data:www-data unless I am missing something (which I probably do).

Thanks.

[EugenMayer]

please do no longer use sync_user, only userid. Does your container really have a user 5000 www-data, with that UID? please ensure you have the newest unison image, since we already fixed a bug in that regard

[ShurikAg]

@EugenMayer I assume that I have the latest one since I just started using it a couple of days ago. Yes, I do have that user with 5000 as an id. So, the instructions from Dockerfile worked fine. I'll try removing the user.

[ShurikAg]

So, I tried using it just with userid

Synced folder looks like this:

root@dcfeba1e2234:/var/www/indago# ls -lsa
total 252
  4 drwxr-xr-x 14 root www-data   4096 Feb  8 14:32 .
  4 drwxr-xr-x  4 root root       4096 Feb  8 02:00 ..
  4 -rwxr-xr-x  1 root www-data     41 Feb  6 23:31 .bowerrc
  4 drwxr-xr-x  7 root www-data   4096 Feb  8 00:10 .git
  4 -rwxr-xr-x  1 root www-data    543 Feb  6 23:31 .gitignore
  4 drwxr-xr-x  4 root www-data   4096 Feb  8 00:13 .idea
  4 -rw-r--r--  1 root www-data   1065 Feb  6 23:31 LICENSE
  4 -rw-r--r--  1 root www-data     40 Feb  6 23:31 README.md
  4 drwxr-xr-x  7 root www-data   4096 Feb  6 23:31 app
  4 drwxr-xr-x  2 root www-data   4096 Feb  6 23:31 bin
  4 -rwxr-xr-x  1 root www-data    682 Feb  6 23:47 bower.json
  4 drwxr-xr-x  3 root www-data   4096 Feb  6 23:31 build
  4 -rwxr-xr-x  1 root www-data   2288 Feb  6 23:31 build.xml
  4 -rwxr-xr-x  1 root www-data   3613 Feb  6 23:31 composer.json
144 -rw-r--r--  1 root www-data 144530 Feb  6 23:31 composer.lock
  4 drwxr-xr-x  2 root www-data   4096 Feb  6 23:31 csv
  4 drwxr-xr-x  3 root www-data   4096 Feb  6 23:31 devops
  4 drwxr-xr-x  4 root www-data   4096 Feb  7 22:27 docker
  4 -rw-r--r--  1 root www-data    155 Feb  6 23:31 docker-compose-dev.yml
  4 -rw-r--r--  1 root www-data    998 Feb  8 00:08 docker-compose.yml
  8 -rw-r--r--  1 root root       5091 Feb  8 14:32 docker-sync.yml
  4 drwxr-xr-x  2 root www-data   4096 Feb  6 23:31 emails
  4 -rw-r--r--  1 root www-data   1498 Feb  6 23:31 indago.iml
  4 drwxr-xr-x  3 root www-data   4096 Feb  6 23:31 src
  4 -rw-r--r--  1 root www-data   2031 Feb  7 23:49 test.sql
  4 drwxr-xr-x 35 root www-data   4096 Feb  6 23:31 vendor
  4 drwxr-xr-x  7 root www-data   4096 Feb  7 01:39 web

/etc/passwd has this:

www-data:x:5000:33:www-data:/var/www:/usr/sbin/nologin

[ShurikAg]

@EugenMayer How can I understand where the problem is or debug it? I am completely stuck.

[EugenMayer]

@ShurikAg well we have different timezones - so i was sleeping :)

First of, in general, it works exactly as you are planing or expecting it to do. Files will end up having UID 5000 in the application container.

I have the strong feeling that you created your sync container once without UID and now reusing it all the time. Please run docker-sync-stack clean or docker-sync clean, then start it again.

If that is not working paste your docker-compose.yml and your full docker-sync.yml

You are most probably having a typo, a indentation issue or something else nasty - nothing realy substantiell is wrong - you already understood it perfectly fine and we all here in docker-sync use it exactly that way :)

[ShurikAg]

So, the one thing that worked is, I removed the forcing of user id and specified 33 in docker-sync.yml. But, who is promising that it is always going to be 33?

As for your last comment, I tried deleting and recreating all containers many times and that behavior didn't change.

[ShurikAg]

Just for the record: docker-sync.yml

options:
  # default: docker-compose.yml if you like, you can set a custom location (path) of your compose file like ~/app/compose.yml
  compose-file-path: './docker-compose.yml'

  # optional, default: docker-compose-dev.yml if you like, you can set a custom location (path) of your compose file. Do not set it, if you do not want to use it at all

  # if its there, it gets used, if you name it explicitly, it HAS to exist
  compose-dev-file-path: './docker-compose-dev.yml'

  # optional, activate this if you need to debug something, default is false
  # IMPORTANT: do not run stable with this, it creates a memory leak, turn off verbose when you are done testing
  verbose: false

  # ADVANCED: default:eugenmayer/rsync -  the image to use for the rsync container. Do not change this until you exactly know, what you are doing
  # optional, default eugenmayer/rsync, you cannot just use "some unison" container, you entrypoint needs features, check the Dockerfiles
  rsync_image: 'eugenmayer/rsync'

  # ADVANCED: default:leighmcculloch/unison -  the image to use for the unison container. Do not change this until you exactly know, what you are doing
  # optional, default eugenmayer/unison, you cannot just use "some unison" container, you entrypoint needs features, check the Dockerfiles
  unison_image: 'eugenmayer/unison'

  # optional, default auto, can be docker-sync, thor or auto and defines how the sync will be invoked on the cli. Mostly depending if your are using docker-sync solo, scaffolded or in development ( thor )
  cli_mode: 'auto'

syncs:
  # IMPORTANT: this name must be unique and should NOT match your real application container name!
  indago-sync:
    # enable terminal_notifier. On every sync sends a Terminal Notification regarding files being synced. ( Mac Only ).
    # good thing in case you are developing and want to know exactly when your changes took effect.
    # be aware in case of unison this only gives you a notification on the initial sync, not the syncs after changes.
    notify_terminal: true

    # which folder to watch / sync from - you can use tilde (~), it will get expanded. Be aware that the trailing slash makes a difference
    # if you add them, only the inner parts of the folder gets synced, otherwise the parent folder will be synced as top-level folder
    src: './'

    # which destination on the sync-container. Since you will use volumes_from to mount this, this should match your code-deployment location in the real container
    dest: '/var/www/indago'

    # when a port of a container is exposed, on which IP does it get exposed. Localhost for docker for mac, something else for docker-machine
    sync_host_ip: '127.0.0.1'

    # should be a unique port this sync instance uses on the host to offer the rsync service on
    sync_host_port: 10871

    # optional, a list of excludes for rsync - see rsync docs for details
    #sync_excludes: ['Gemfile.lock', 'Gemfile', 'config.rb', '.sass-cache/', 'sass/', 'sass-cache/', 'composer.json' , 'bower.json', 'package.json', 'Gruntfile*', 'bower_components/', 'node_modules/', '.gitignore', '.git/', '*.coffee', '*.scss', '*.sass']

    # use this to change the exclude syntax.
    # Path: you match the exact path ( nesting problem )
    # Name: If a file or a folder does match this string ( solves nesting problem )
    # Regex: Define a regular expression
    # none: You can define a type for each sync exclude, so sync_excludes: ['Name .git', 'Path Gemlock']
    #
    # for more see http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#pathspec
    #sync_excludes_type: 'Path'

    # optional: use this to switch to rsync verbose mode
    #sync_args: '-v'

    # optional, a list of regular expressions to exclude from the fswatch - see fswatch docs for details

    # optional, default can be either rsync or unison See Strategies in the wiki for explanation
    sync_strategy: 'unison'

    # optional, use this to map the files to a specific user on sync. Helps fixing permissions. You can use names and uids
    #sync_user: 'www-data'

    # this does not user groupmap but rather configures the server to map
    # optional: usually if you map users you want to set the user id of your application container here
    sync_userid: '33'

    # optional, use this to map the files to a specific group on sync. Helps fixing permissions, You can use names and guids
#    sync_group: 'www-data'
#    sync_uid: 5000

    # optional: usually if you map groups you want to set the group id of your application container here
    # this does not user groupmap but rather configures the server to map
    #sync_groupid: '6000'

    #watch_excludes: ['.*/.git', '.*/node_modules', '.*/bower_components', '.*/sass-cache', '.*/.sass-cache', '.*/.sass-cache', '.coffee', '.scss', '.sass', '.gitignore']

    # optional: use this to switch to fswatch verbose mode
    watch_args: '-v'

    # optional: default is fswatch, if set to disable, no watcher will be used and you would need to start the sync manually
    watch_strategy: 'fswatch'

and cocker-sompose.yml

version: "2"
services:
  mysql:
    image: mysql:5.6
    container_name: mysql
    volumes:
      - ~/DevShit/PRIZ/mysql-data:/var/lib/mysql
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
    ports:
      - "3306:3306"
  memcached:
    image: memcached
    container_name: memcached
  web:
    build: ./docker/
    container_name: indago_web
    depends_on:
      - mysql
      - memcached
    links:
      - mysql:mysql
      - memcached:memcached
    volumes:
      - ./docker/php-fpm/www.conf:/usr/local/etc/php-fpm.d/www.conf
#      - ./:/var/www/indago
    environment:
      ING_DB_USER: priz_wp
      ING_DB_PASSWORD: Shurik_0(8
      ING_DB_NAME: priz_wp
      ING_DB_HOST: mysql
  proxy:
    image: nginx
    container_name: indago_proxy
    depends_on:
      - web
    links:
      - web:web
    ports:
      - "80:80"
    volumes:
      - ./docker/web/nginx.conf:/etc/nginx/nginx.conf
      - ./docker/web/default.conf:/etc/nginx/conf.d/default.conf
      - ./:/var/www/indago

[EugenMayer]

Please remove all comments, the whole options section, sync_host_ip: '127.0.0.1' and sync_host_port, remove watch_strategy: 'fswatch' and watch_args: '-v'

please use a proper boilerplate like this https://github.com/EugenMayer/docker-sync-boilerplate/blob/master/unison-ftp-user/docker-sync.yml as a starting point - this should fix your issue.

[ShurikAg]

@EugenMayer I'll give it a try (I simply followed the docs :)

[EugenMayer]

did you ? :) https://goo.gl/7XvYtD

[ShurikAg]

Yes I did: https://github.com/EugenMayer/docker-sync/wiki/2.-Configuration#docker-syncyml

[ShurikAg]

So, now I have this:

options:
  verbose: true
syncs:
  #IMPORTANT: ensure this name is unique and does not match your other application container name
  indago-sync: # tip: add -sync and you keep consistent names als a convention
    src: './'
    dest: '/var/www/indago'
    sync_strategy: 'unison'
    #sync_excludes: ['ignored_folder', '.ignored_dot_folder']
    #sync_excludes_type: 'Name'
    # this is just for test purposes, do not use sync_user anymore, it will be deprecated, just used uid
    #sync_user: 'specialone'
    # example on how to let the files be owned by uid 22, thats the ftp user
    sync_uid: '33'

Same picture as before. Ownership is set to root.

[EugenMayer]

well there is a link to the boilerplate also, nevertheless i understand how you got confused, i added https://goo.gl/1uzztX to the configuration reference ,so its clear its a full reference, not an example.

[EugenMayer]

did you CLEAN the stack before you tested the new configuration? That is very important. That said, after cleaning, all you containers should be gone, ensure with docker ps and docker ps -a ensure there is no leftover, otherwise the older container gets reused.

Please paste the first 10 lines of the startup here (of docker sync )

[ShurikAg]

Yes, I did run clean. But all containers apparently remained in place.

$ docker-sync-stack clean
          ok  Found implicit docker-compose-dev.yml and using it from /Users/ShurikAg/Dev/indago/indago/docker-compose-dev.yml
Service "web" mounts volumes from "indago-sync", which is not the name of a service or container.
/Library/Ruby/Gems/2.0.0/gems/docker-compose-1.0.4/lib/docker/compose/session.rb:238:in `run!': 'down' failed with status 'pid 73515 exit 1': Service "web" mounts volumes from "indago-sync", which is not the name of a service or container. (Docker::Compose::Error)
    from /Library/Ruby/Gems/2.0.0/gems/docker-compose-1.0.4/lib/docker/compose/session.rb:92:in `down'
    from /Library/Ruby/Gems/2.0.0/gems/docker-sync-0.1.5/lib/docker-sync/compose.rb:57:in `clean'
    from /Library/Ruby/Gems/2.0.0/gems/docker-sync-0.1.5/tasks/stack/stack.thor:87:in `clean'
    from /Library/Ruby/Gems/2.0.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
    from /Library/Ruby/Gems/2.0.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
    from /Library/Ruby/Gems/2.0.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
    from /Library/Ruby/Gems/2.0.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
    from /Library/Ruby/Gems/2.0.0/gems/docker-sync-0.1.5/bin/docker-sync-stack:26:in `<top (required)>'
    from /usr/local/bin/docker-sync-stack:23:in `load'
    from /usr/local/bin/docker-sync-stack:23:in `<main>'

and if I start it now, I get:

$ docker-sync-stack start
          ok  Starting unison
          ok  indago-sync container not running
          ok  creating indago-sync container
     command  docker run -p '127.0.0.1::5000'                               -v indago-sync:/var/www/indago                               -e UNISON_DIR=/var/www/indago                               -e TZ=${TZ-`readlink /etc/localtime | sed -e 's,/usr/share/zoneinfo/,,'`}                               -e UNISON_EXCLUDES="" -e UNISON_OWNER_UID="33"                                                              --name indago-sync                               -d eugenmayer/unison:unox
          ok  starting initial sync of indago-sync
     command  docker inspect --format=" {{ .NetworkSettings.Ports }} " indago-sync | /usr/bin/sed  -E "s/.*map\[5000[^ ]+ ([0-9]*)[^0-9].*/\1/"
     command  unison /Users/ShurikAg/Dev/indago/indago -auto -batch socket://127.0.0.1:32817
       error  Error starting sync, exit code 0
     message  Warning: No archive files were found for these roots, whose canonical names are:
    /Users/ShurikAg/Dev/indago/indago
    //b2d7ad0a4210//var/www/indago

But, it seem to work (somehow)

[EugenMayer]

Probably now and sync_host_ip: 127.0.0.1 and stop it all, clean and probably restart, so you remove all the dangling containers. the problem is when you change docker-sync file while the stack is running, there is a potential that you do later no longer reference the correct container.

Glad it works now though

[ShurikAg]

Thanks

[ShurikAg]

@EugenMayer One note through. The correct ownership was set only with

sync_userid: '33'

not

sync_uid: '33'

[EugenMayer]

thats a good catch! fixed that in the docs