lchown permission denied (non-root user)

GRX commented 4 years ago

Hi,

I'm trying to get the sshfs volume working for my backup stack, but everytime I try to mount the endpoint I get prompted with permission denied.

With the user that's running docker-compose, I can access the folder so I'm not sure why this is going wrong.

Error when running docker-compose: Error response from daemon: failed to copy file info for /var/lib/docker/plugins/a1318307eab6ed3e2bcfffa7a5226d557e7a2a6f12b4d48113d2e79063da848d/propagated-mount/e6c0b9b0593381f295d4a5c925f8ad8b: failed to chown /var/lib/docker/plugins/a1318307eab6ed3e2bcfffa7a5226d557e7a2a6f12b4d48113d2e79063da848d/propagated-mount/e6c0b9b0593381f295d4a5c925f8ad8b: lchown /var/lib/docker/plugins/a1318307eab6ed3e2bcfffa7a5226d557e7a2a6f12b4d48113d2e79063da848d/propagated-mount/e6c0b9b0593381f295d4a5c925f8ad8b: permission denied.

andanotherusername commented 4 years ago

You can workaround this issue by leaving a dump file on the remote location.

sbriels commented 4 years ago

I didnt try it yet with docker compose, but I also couldn't make any connections. I finally solved it by:

installing the plugin as: docker plugin install --grant-all-permissions vieux/sshfs sshkey.source=/root/.ssh/
setting up public/private key for root (sudo -i , then ssh-keygen -b 4096 -t rsa )
setting up the volume: docker volume create -d vieux/sshfs -o sshcmd=user@192.168.1.1:/home/user/ sshvolume

Hope this helps either you or the development team.

chrlutz commented 3 years ago

I'm getting this error message as well:

# docker run -ti -v nextcloud-html:/var/www/html nextcloud /bin/bash
docker: Error response from daemon: failed to copy file info for /var/lib/docker/plugins/2bb483b4a3521a55bff3d7284d4f1646b74092430c307ba7fea29306ea92fa89/propagated-mount/21558aea1829ef2d4bd4c0b95f4e5f60: failed to chown /var/lib/docker/plugins/2bb483b4a3521a55bff3d7284d4f1646b74092430c307ba7fea29306ea92fa89/propagated-mount/21558aea1829ef2d4bd4c0b95f4e5f60: lchown /var/lib/docker/plugins/2bb483b4a3521a55bff3d7284d4f1646b74092430c307ba7fea29306ea92fa89/propagated-mount/21558aea1829ef2d4bd4c0b95f4e5f60: permission denied.
See 'docker run --help'.

In my case this seems to happen because I have use two sshfs volumes (nextcloud-db and nextcloud-html) on the same remote ssh server like in this example:

# docker volume inspect nextcloud-db 
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "vieux/sshfs:latest",
        "Labels": null,
        "Mountpoint": "/mnt/volumes/2e01dcda104ec5521369b7d308810632",
        "Name": "nextcloud-db",
        "Options": {
            "allow_other": "",
            "gid": "999",
            "password": "xxx",
            "sshcmd": "uxxx-sub1@uxxx.your-storagebox.de:nextcloud_db",
            "uid": "999"
        },
        "Scope": "local"
    }
]

# docker volume inspect nextcloud-html
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "vieux/sshfs:latest",
        "Labels": null,
        "Mountpoint": "/mnt/volumes/21558aea1829ef2d4bd4c0b95f4e5f60",
        "Name": "nextcloud-html",
        "Options": {
            "allow_other": "",
            "password": "xxx",
            "sshcmd": "uxxx-sub2@uxxx.your-storagebox.de:",
            "uid": "33"
        },
        "Scope": "local"
    }
]

I figured out that it seems I cannot use both volumes at the same time.

It is possible to mount volume nextcloud-db to a mariadb container, but if I startup a second container (see docker run in my 3rd Line) this happens. If I stop/rm the mariadb container first, my 3rd line executes successfully.

Please note that both volumes are hosted on the same host (but accessed via different ssh-users). Could it be related to the different uid-Values I use in the volumes?

best regards, Christoph

chrlutz commented 3 years ago

You can workaround this issue by leaving a dump file on the remote location.

What exactly do you mean by that? Do you have got an example? How can I produce such a dump file?

blackfeather9 commented 3 years ago

I am having the same issue but may have a solution. Trying to use docker-compose to attach sshfs volumes.

docker version: 20.10.7 host: ubuntu 20.04 (kernel 5.4.0-80-generic)

Tried with public key authentication as well as password. Volume is created, user has full r/w/x over ssh with either method, can manually edit. Run docker-compse and watch auth logs on storage server, no problems there. Removed ACLs, set remote dirs to 777 and the parent folders.

Every time after the volume driver authenticates, docker-compose was failing with the following error:

failed to copy file info for /var/lib/docker/plugins/33378cc08f72a307a93f8f14391ef77eb78de6ed6f97fc427a3c096d0f162956/propagated-mount/364cea55c25d60e3da22a307ac46c260: failed to chown /var/lib/docker/plugins/33378cc08f72a307a93f8f14391ef77eb78de6ed6f97fc427a3c096d0f162956/propagated-mount/364cea55c25d60e3da22a307ac46c260: lchown /var/lib/docker/plugins/33378cc08f72a307a93f8f14391ef77eb78de6ed6f97fc427a3c096d0f162956/propagated-mount/364cea55c25d60e3da22a307ac46c260: permission denied

Tried above suggestions, including creating the volume in advance, and the suggestions in issues 17, 58. 65.

Possible solution: the "dump" file mentioned by @andanotherusername

Steps to Recreate:

create your docker vol by hand or using docker-compose
lchown fails with permission denied
connect to storage target over ssh and cd into the volume dir, i.e. /mnt/volumes/containers/mycontainer
create a dump file, i.e. touch dump -- that's it, it doesn't even have to be called dump.
run your container/compose file again and watch as the volume successfully mounts

This did not work when I set custom uid/gid in my compose file, but with the defaults it seems to be successful and the container can now mount + write to the sshfs volume!

But isn't this still an issue, because without manually creating files within an sshfs volume, they will not mount? This will break any docker automation unless you create scripts to stage your volume storage in advance

vieux / docker-volume-sshfs

lchown permission denied (non-root user) #76