docker: Error response from daemon: VolumeDriver.Mount

[MileanCo]

I'm getting an error after creating my docker volumes. Steps taken: 1) sudo netappdvp --config=/etc/netappdvp/ontap-nas.json

My config file includes the MGMT IP and data LIF IP for my VSIM filer. I can see the volumes here:

[user@scspr0135851001 ~]$ docker volume ls
list netapp-nas: Post http://%2Frun%2Fdocker%2Fplugins%2Fnetapp-nas.sock/VolumeDriver.List: dial unix /run/docker/plugins/netapp-nas.sock: connect: connection refused
DRIVER              VOLUME NAME
netapp              ndvp_2

VSIM:

user-vsim11472669096::> vol show
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
jstenkvi-vsim1
          vol0         aggr0_jstenkvi_vsim1_0
                                    online     RW       3.57GB    170.1MB   95%
jstenkvi-vsim2
          vol0         aggr0        online     RW       3.57GB    361.3MB   90%
vs0       netappdvp_ndvp_2
                       aggr1        online     RW          1GB    972.6MB    5%
vs0       svm_root     aggr1        online     RW         20MB    18.29MB    8%
vs0       vol1         aggr1        online     RW        100MB    84.34MB   15%

2) docker volume create -d netapp --name ndvp_1

Fails on 3rd step:

[user@scspr0135851001 ~]$ sudo docker run --rm -it --volume-driver netapp --volume ndvp_2:/my_vol alpine ash
docker: Error response from daemon: VolumeDriver.Mount: Error checking if /var/lib/docker-volumes/netapp/netappdvp_ndvp_2 is already mounted: exit status 1.

My log /var/log/netappdvp/netapp.log looks like:

^[[37mDEBU^[[0m[0339] Listing volumes...
^[[37mDEBU^[[0m[0339] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0339] List() adding volume: ndvp_2 from: /var/lib/docker-volumes/netapp/netappdvp_ndvp_2
^[[37mDEBU^[[0m[0339] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0339] List() adding volume: ndvp_1 from: /var/lib/docker-volumes/netapp/netappdvp_ndvp_1
^[[37mDEBU^[[0m[0356] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0356] Getting path for volume 'netappdvp_ndvp_1' as '/var/lib/docker-volumes/netapp/netappdvp_ndvp_1'
^[[37mDEBU^[[0m[0356] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0356] Mounting volume netappdvp_ndvp_1 on /var/lib/docker-volumes/netapp/netappdvp_ndvp_1
^[[37mDEBU^[[0m[0356] Begin osutils.GetDFOutput
^[[31mERRO^[[0m[0356] Error encountered gathering df output: exit status 1
^[[37mDEBU^[[0m[0356] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0356] Unmounting docker volume netappdvp_ndvp_1
^[[37mDEBU^[[0m[0356] OntapNASStorageDriver#Detach(netappdvp_ndvp_1, /var/lib/docker-volumes/netapp/netappdvp_ndvp_1)
^[[37mDEBU^[[0m[0356] cmd==umount /var/lib/docker-volumes/netapp/netappdvp_ndvp_1
^[[37mDEBU^[[0m[0356] out==umount: /var/lib/docker-volumes/netapp/netappdvp_ndvp_1: not mounted

^[[37mDEBU^[[0m[0362] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0362] Getting path for volume 'netappdvp_ndvp_2' as '/var/lib/docker-volumes/netapp/netappdvp_ndvp_2'
^[[37mDEBU^[[0m[0362] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0362] Mounting volume netappdvp_ndvp_2 on /var/lib/docker-volumes/netapp/netappdvp_ndvp_2
^[[37mDEBU^[[0m[0362] Begin osutils.GetDFOutput
^[[31mERRO^[[0m[0362] Error encountered gathering df output: exit status 1
^[[37mDEBU^[[0m[0362] storagePrefix is unspecified, using default prefix: netappdvp_
^[[37mDEBU^[[0m[0362] Unmounting docker volume netappdvp_ndvp_2
^[[37mDEBU^[[0m[0362] OntapNASStorageDriver#Detach(netappdvp_ndvp_2, /var/lib/docker-volumes/netapp/netappdvp_ndvp_2)
^[[37mDEBU^[[0m[0362] cmd==umount /var/lib/docker-volumes/netapp/netappdvp_ndvp_2
^[[37mDEBU^[[0m[0362] out==umount: /var/lib/docker-volumes/netapp/netappdvp_ndvp_2: not mounted

[bnaylor]

Some background traffic turned this up as being mainly config issues. Closing.

[MileanCo]

Apparently this is an error/bug caused by running the plugin with a different driver ID and that left a socket file that was causing the nDVP to error out when it issued a ‘df’: ERRO[3077] Error encountered gathering df output: exit status 1

Fixed this by removing the old .sock files in /var/run/docker and /var/lib/docker and restarting Docker

Fixed in this commit : https://github.com/NetApp/netappdvp/commit/b4c05c03aba8d54bb763326c83e23f50f5bedead

[primun]

I am facing the similar issue while i use docker compose [root@scsor0009368001 home]# docker-compose up Creating volume "home_clusterhome1" with netapp driver Creating volume "home_mysqldata1" with netapp driver home_nginx_1 is up-to-date Creating home_mysqldb_1

ERROR: for mysqldb Cannot create container for service mysqldb: VolumeDriver.Mount: Error checking if /var/lib/docker-volumes/netapp/home_mysqldata1 is already mounted: exit status 1 ERROR: Encountered errors while bringing up the project.

then i go and remove the socks file as stated above and start encountering the below issue: [root@scsor0009368001 plugins]# rm netapp.sock rm: remove socket `netapp.sock'? y [root@scsor0009368001 home]# systemctl restart docker

[root@scsor0009368001 home]# docker-compose up Creating volume "home_clusterhome1" with netapp driver ERROR: Volume clusterhome1 specifies nonexistent driver netapp

i check if netapp ndvp is running: [root@scsor0009368001 home]# netappdvp --config=/etc/netappdvp/ontap-nas.json & [3] 665 [root@scsor0009368001 home]# INFO[2017-03-14T10:09:31-04:00] Successfully initialized Ontap NAS Docker driver version 1.3.3 [native] Logfile Location (Level: info): /var/log/netappdvp/netapp.log

[root@scsor0009368001 home]# docker-compose up Starting home_nginx_1 Creating home_mysqldb_1

ERROR: for mysqldb Cannot create container for service mysqldb: VolumeDriver.Mount: Error checking if /var/lib/docker-volumes/netapp/home_mysqldata1 is already mounted: exit status 1 ERROR: Encountered errors while bringing up the project.

[primun]

var/log output attached for ref: [root@scsor0009368001 netappdvp]# vi netapp.log ^[[34mINFO^[[0m[2017-03-14T07:12:33-04:00] Using storage driver: ontap-nas ^[[34mINFO^[[0m[2017-03-14T07:12:33-04:00] Using config: {1 ontap-nas false map[] false [34 34] []} ^[[34mINFO^[[0m[2017-03-14T07:12:33-04:00] Starting docker volume plugin with the following options: ^[[34mport^[[0m= ^[[34mvolume-driver^[[0m=netapp ^[[34mvolumeDir^[[0m="/var/lib/docker-volumes/netapp" ^[[31mERRO^[[0m[2017-03-14T07:54:35-04:00] Error encountered gathering df output: exit status 1 ^[[34mINFO^[[0m[2017-03-14T09:09:26-04:00] Using storage driver: ontap-nas ^[[34mINFO^[[0m[2017-03-14T09:09:26-04:00] Using config: {1 ontap-nas false map[] false [34 34] []} ^[[34mINFO^[[0m[2017-03-14T09:09:26-04:00] Starting docker volume plugin with the following options: ^[[34mport^[[0m= ^[[34mvolume-driver^[[0m=netapp ^[[34mvolumeDir^[[0m="/var/lib/docker-volumes/netapp" ^[[31mERRO^[[0m[2017-03-14T09:19:27-04:00] Error encountered gathering df output: exit status 1 ^[[31mERRO^[[0m[2017-03-14T09:26:18-04:00] Error encountered gathering df output: exit status 1 ^[[34mINFO^[[0m[2017-03-14T09:40:41-04:00] Using storage driver: ontap-nas ^[[34mINFO^[[0m[2017-03-14T09:40:41-04:00] Using config: {1 ontap-nas false map[] false [34 34] []} ^[[34mINFO^[[0m[2017-03-14T09:40:41-04:00] Starting docker volume plugin with the following options: ^[[34mport^[[0m= ^[[34mvolume-driver^[[0m=netapp ^[[34mvolumeDir^[[0m="/var/lib/docker-volumes/netapp" ^[[31mERRO^[[0m[2017-03-14T09:41:17-04:00] Error encountered gathering df output: exit status 1 ^[[34mINFO^[[0m[2017-03-14T09:46:41-04:00] Using storage driver: ontap-nas ^[[34mINFO^[[0m[2017-03-14T09:46:41-04:00] Using config: {1 ontap-nas false map[] false [34 34] []} ^[[34mINFO^[[0m[2017-03-14T09:46:41-04:00] Starting docker volume plugin with the following options: ^[[34mport^[[0m= ^[[34mvolume-driver^[[0m=netapp ^[[34mvolumeDir^[[0m="/var/lib/docker-volumes/netapp" ^[[31mERRO^[[0m[2017-03-14T09:47:20-04:00] Error encountered gathering df output: exit status 1 ^[[31mERRO^[[0m[2017-03-14T09:53:30-04:00] Error encountered gathering df output: exit status 1 ^[[34mINFO^[[0m[2017-03-14T10:09:31-04:00] Using storage driver: ontap-nas ^[[34mINFO^[[0m[2017-03-14T10:09:31-04:00] Using config: {1 ontap-nas false map[] false [34 34] []} ^[[34mINFO^[[0m[2017-03-14T10:09:31-04:00] Starting docker volume plugin with the following options: ^[[34mport^[[0m= ^[[34mvolume-driver^[[0m=netapp ^[[34mvolumeDir^[[0m="/var/lib/docker-volumes/netapp" ^[[31mERRO^[[0m[2017-03-14T10:11:48-04:00] Error encountered gathering df output: exit status 1 ^[[31mERRO^[[0m[2017-03-14T10:13:59-04:00] Error encountered gathering df output: exit status