Unable to create vsphere volume in swarm

[velislav]

I am unable to create volume in swarm

$ docker version
Client:
 Version:      17.11.0-ce
 API version:  1.34
 Go version:   go1.8.5
 Git commit:   1caf76c
 Built:        Mon Nov 20 18:36:39 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.11.0-ce
 API version:  1.34 (minimum version 1.12)
 Go version:   go1.8.5
 Git commit:   1caf76c
 Built:        Mon Nov 20 18:35:09 2017
 OS/Arch:      linux/amd64
 Experimental: false

Plugins installed on all servers:

$ docker plugin ls
ID                  NAME                DESCRIPTION                           ENABLED
c1cbd66b7793        vsphere:latest      VMWare vSphere Docker Volume plugin   true
10485bd0f7e9        vfile:latest        VMWare vFile Docker Volume plugin     true

With --driver=vfile

$ docker volume create --driver=vfile --name=for_delete2 -o size=1gb
Error response from daemon: create for_delete2: VolumeDriver.Create: Failed to create internal volume for_delete2. Reason: Error response from daemon: {"message":"create _vF_for_delete2: VolumeDriver.Create: Device not found"}. Check the status of the volumes belonging to driver "vsphere"

Logs from /var/log/vfile.log

2017-12-08 16:17:30.440450093 +0000 UTC [INFO] VolumeDriver Get: for_delete2
2017-12-08 16:17:30.482313685 +0000 UTC [INFO] Volume not found: for_delete2
2017-12-08 16:17:30.48242825 +0000 UTC [ERROR] Failed to get volume meta-data name="for_delete2" error="No such volume" 
2017-12-08 16:17:30.482845028 +0000 UTC [INFO] VolumeDriver Create: for_delete2
2017-12-08 16:17:30.482890453 +0000 UTC [INFO] Attempting to write initial metadata entry for for_delete2
2017-12-08 16:17:30.531696167 +0000 UTC [INFO] Attempting to create internal volume for for_delete2
2017-12-08 16:17:31.905202777 +0000 UTC [WARNING] Failed to create internal volume for_delete2. Reason: Error response from daemon: {"message":"create _vF_for_delete2: VolumeDriver.Create: Device not found"}. Check the status of the volumes belonging to driver "vsphere".

Logs from /var/log/docker-volume-vsphere.log

2017-12-08 16:17:30.796913759 +0000 UTC [ERROR] Failed to get volume meta-data name="_vF_for_delete2" error="Volume _vF_for_delete2 not found (file: /vmfs/volumes/Netapp vol2_2/dockvols/_DEFAULT/_vF_for_delete2.vmdk)" 
2017-12-08 16:17:31.015726382 +0000 UTC [INFO] Attaching volume and creating filesystem name="_vF_for_delete2" fstype=ext4 
2017-12-08 16:17:31.490620603 +0000 UTC [WARNING] Get device path failed for unit %s @ PCI bridge device %s: 0Error="open /sys/bus/pci/devices/0000:00:/pci_bus: no such file or directory" 
2017-12-08 16:17:31.490738955 +0000 UTC [ERROR] Failed to get device path volDev={0 192 } err="Device not found" 
2017-12-08 16:17:31.490793861 +0000 UTC [ERROR] Could not find attached device, removing the volume error="Device not found" name="_vF_for_delete2"

With --driver=vsphere

$ docker volume create --driver=vsphere --name=for_delete -o size=1gb
Error response from daemon: create for_delete: VolumeDriver.Create: Device not found

Logs from /var/log/docker-volume-vsphere.log

2017-12-08 16:28:25.626923555 +0000 UTC [ERROR] Failed to get volume meta-data error="Volume for_delete not found (file: /vmfs/volumes/Netapp vol2_2/dockvols/_DEFAULT/for_delete.vmdk)" name="for_delete" 
2017-12-08 16:28:25.838294643 +0000 UTC [INFO] Attaching volume and creating filesystem fstype=ext4 name="for_delete" 
2017-12-08 16:28:26.264237615 +0000 UTC [WARNING] Get device path failed for unit %s @ PCI bridge device %s: 0Error="open /sys/bus/pci/devices/0000:00:/pci_bus: no such file or directory" 
2017-12-08 16:28:26.264377056 +0000 UTC [ERROR] Failed to get device path volDev={0 192 } err="Device not found" 
2017-12-08 16:28:26.264450983 +0000 UTC [ERROR] Could not find attached device, removing the volume name="for_delete" error="Device not found"

devices

$ ls '/sys/bus/pci/devices/0000:00:/pci_bus'
ls: cannot access '/sys/bus/pci/devices/0000:00:/pci_bus': No such file or directory
$ ls /sys/bus/pci/devices/
0000:00:00.0  0000:00:07.3  0000:00:11.0  0000:00:15.3  0000:00:15.7  0000:00:16.3  0000:00:16.7  0000:00:17.3  0000:00:17.7  0000:00:18.3  0000:00:18.7
0000:00:01.0  0000:00:07.7  0000:00:15.0  0000:00:15.4  0000:00:16.0  0000:00:16.4  0000:00:17.0  0000:00:17.4  0000:00:18.0  0000:00:18.4  0000:02:01.0
0000:00:07.0  0000:00:0f.0  0000:00:15.1  0000:00:15.5  0000:00:16.1  0000:00:16.5  0000:00:17.1  0000:00:17.5  0000:00:18.1  0000:00:18.5  0000:03:00.0
0000:00:07.1  0000:00:10.0  0000:00:15.2  0000:00:15.6  0000:00:16.2  0000:00:16.6  0000:00:17.2  0000:00:17.6  0000:00:18.2  0000:00:18.6  0000:0b:00.0

[govint]

@velislav, the 0.19 release of the plugin uses an enhanced approach to detect container volumes in the Linux host. Please install the 0.19 version VIB on the ESX host let us know if that works fine.

Unless otherwise noted in the release notes, its preferred to update both the plugin and the VIB to matching versions.

[shuklanirdesh82]

duplicate to https://github.com/vmware/docker-volume-vsphere/issues/1969 .. @velislav As @govint mentioned please update your VIB to 0.19 release and your problem should be resolved

[velislav]

Thanks, 0.19 version VIB on the ESX host fixed the problem.