VolumeDriver.Mount: error mounting <volume>: exit status 1

[jgraeve]

I have 3 centos 7 nodes with gluster installed

[root@ds01 ~]# gluster volume info Volume Name: gv0 Type: Replicate Volume ID: 98f1c6e3-8fa0-4a2b-93d5-95b64435eb04 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: ds01:/data/brick1/gv0 Brick2: ds02:/data/brick1/gv0 Brick3: ds03:/data/brick1/gv0 Options Reconfigured: transport.address-family: inet storage.fips-mode-rchecksum: on nfs.disable: on performance.client-io-threads: off

The gv0 gluster volume is working fine. When I mount it and create files they are replicated to ds01,ds02,ds03

But when I create a docker volume with the glusterfs plugin and try to mount it in a docker instance I get VolumeDriver.Mount: error mounting : exit status 1

All help appreciated!

versions used: docker version 19.03.12 glusterfs 7.6 centos 7 Linux ds01 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[root@ds01 ~]# docker plugin install --alias glusterfs trajano/glusterfs-volume-plugin --grant-all-permissions --disable latest: Pulling from trajano/glusterfs-volume-plugin a392984758fa: Download complete Digest: sha256:f054d37e71aca82426c12c0d4a555861e1225969df2df6c45641303b4cf18a95 Status: Downloaded newer image for trajano/glusterfs-volume-plugin:latest Installed plugin trajano/glusterfs-volume-plugin [root@ds01 ~]# docker plugin set glusterfs SERVERS=ds01,ds02,ds03 [root@ds01 ~]# docker plugin enable glusterfs glusterfs [root@ds01 ~]# docker plugin list ID NAME DESCRIPTION ENABLED e6ee819ceeca glusterfs:latest GlusterFS plugin for Docker true [root@ds01 ~]# docker plugin inspect glusterfs [ { "Config": { "Args": { "Description": "", "Name": "", "Settable": null, "Value": null }, "Description": "GlusterFS plugin for Docker", "DockerVersion": "18.03.0-ce", "Documentation": "https://github.com/trajano/docker-volume-plugins/", "Entrypoint": [ "/glusterfs-volume-plugin" ], "Env": [ { "Description": "", "Name": "SERVERS", "Settable": [ "value" ], "Value": "" } ], "Interface": { "Socket": "gfs.sock", "Types": [ "docker.volumedriver/1.0" ] }, "IpcHost": false, "Linux": { "AllowAllDevices": false, "Capabilities": [ "CAP_SYS_ADMIN" ], "Devices": [ { "Description": "", "Name": "", "Path": "/dev/fuse", "Settable": null } ] }, "Mounts": null, "Network": { "Type": "host" }, "PidHost": false, "PropagatedMount": "/var/lib/docker-volumes", "User": {}, "WorkDir": "", "rootfs": { "diff_ids": [ "sha256:9e8240f5b99231266ccb3260422fa333d6f72b3d7f6c77042d775ffc0e89b9ba" ], "type": "layers" } }, "Enabled": true, "Id": "e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887", "Name": "glusterfs:latest", "PluginReference": "docker.io/trajano/glusterfs-volume-plugin:latest", "Settings": { "Args": [], "Devices": [ { "Description": "", "Name": "", "Path": "/dev/fuse", "Settable": null } ], "Env": [ "SERVERS=ds01,ds02,ds03" ], "Mounts": [] } } ] [root@ds01 ~]# docker volume create -d glusterfs gv2 gv2 [root@ds01 ~]# docker run -it -v gv2:/mnt alpine docker: Error response from daemon: VolumeDriver.Mount: error mounting gv2: exit status 1. See 'docker run --help'.

Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:28+02:00" level=error msg="Entering go-plugins-helpers capabilitiesPath" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=error msg="Entering go-plugins-helpers getPath" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=error msg="Entering go-plugins-helpers capabilitiesPath" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=error msg="Entering go-plugins-helpers getPath" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=error msg="Entering go-plugins-helpers capabilitiesPath" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=error msg="Entering go-plugins-helpers mountPath" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=error msg="[-s ds01 -s ds02 -s ds03 --volfile-id=gv2 /var/lib/docker-volumes/de3b2d20f02f69bf4fd0112157e8ac2a259e820a06610b906d20693bb091241b]" plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29+02:00" level=info msg="Command output: " plugin=e6ee819ceeca73104da53ae7cc9b16150ddef6536f0dfbf7bcc282f8915b6887 Jul 10 14:57:29 ds01 dockerd: time="2020-07-10T14:57:29.059610710+02:00" level=error msg="Handler for POST /v1.40/containers/create returned error: VolumeDriver.Mount: error mounting gv2: exit status 1"

[goodwolf]

The latest tag is the old version, install the plugin with the tag v2.0.3

docker plugin install --alias glusterfs trajano/glusterfs-volume-plugin:v2.0.3 --grant-all-permissions --disable

[pmorch]

I had the exact same problem, and @goodwolf's answer fixed it.

It is a problem that latest (the default) is older than 2.0.3 and that latest doesn't work. @jgraeve , did this fix your problem too. (Or have you moved on to other things and can't test it? That is a valid real-life answer too...)

[wiryonolau]

It doesn't work even when using 2.0.3, same error message

[crmillner]

I think that instead of: docker volume create -d glusterfs gv2

it should be: docker volume create -d glusterfs gv0

I think that the volume name must match the gluster volume name. At least that seemed to work for me when I got this error.

[wiryonolau]

So we need to create the volume name in the gluster first if we want to use specific name ? If we have multiple stack it will be bad to use same volume name

I think mounting glusterfs first to the os then use local-persist docker driver is more flexible.