[VVOL] Clone of a volume from existing volume fails for vvol datastore

[ashahi1]

Steps:

1. Create a volume on vvol datastore.
2. Clone the volume - error: VolumeDriver.Create: Server returned an error: (vmodl.fault.InvalidArgument)

Logs have been attached at the end.

---

Detailed steps are as follows:

1. Created a volume on a vvol datastore - MyVolume

   
   root@sc-rdops-vm02-dhcp-52-237:~# docker volume create --driver=vsphere --name=MyVolume -o size=100mb
   MyVolume
   root@sc-rdops-vm02-dhcp-52-237:~# docker volume ls
   DRIVER              VOLUME NAME
   vsphere:latest      MyVolume@cokeshared

[root@sc-rdops-vm01-dhcp-24-159:~] esxcli storage guestvol volume ls Volume Datastore VMGroup Capacity Used Filesystem Policy Disk Format Attached-to Access Attach-as Created By Created Date

---

MyVolume cokeshared _DEFAULT 100MB 6MB ext4 N/A thin detached read-write independent_persistent ubuntu-VM1.3 Wed Nov 22 00:59:07 2017 [root@sc-rdops-vm01-dhcp-24-159:~]

2. Tried to clone a existing volume but operation failed

root@sc-rdops-vm02-dhcp-52-237:~# docker volume create --driver=vsphere --name=CloneVolume -o clone-from=MyVolume Error response from daemon: create CloneVolume: VolumeDriver.Create: Server returned an error: (vmodl.fault.InvalidArgument) { dynamicType = , dynamicProperty = (vmodl.DynamicProperty) [], msg = 'A specified parameter was not correct: ', faultCause = , faultMessage = (vmodl.LocalizableMessage) [], invalidProperty = } root@sc-rdops-vm02-dhcp-52-237:~#

Logs:
[docker-volume-vsphere.txt](https://github.com/vmware/docker-volume-vsphere/files/1493565/docker-volume-vsphere.txt)
[vmdk_ops.1.txt](https://github.com/vmware/docker-volume-vsphere/files/1493566/vmdk_ops.1.txt)
[vmdk_ops.txt](https://github.com/vmware/docker-volume-vsphere/files/1493567/vmdk_ops.txt)

[tusharnt]

@govint can you please update the issue with your analysis of the root cause?

[govint]

The clone op is presently unsupported for vvol based container volumes and will be fixed soon.

[govint]

The root cause seems to be with how the server on the EX host updates the volume KV. With vvols the KV sidecar is itself a vvol and the sidecar file is essentially a descriptor and that data gets overwritten when the KV is updated. While it works fine to open and read/write the sidecar on vmfs and vsan, this doesn't work on vvols as the array side vvol needs to be bound to the host before IO can be done on it. Will post a PR with a backend friendly IO interface than what's used now. This will then apply to all volume backends.

[govint]

Tried out doing native APIs to read/write the KV files, this doesn't seem to work via the python interface to the platform DLLs. Have switched to using regular files for vvol based volumes. Modified to create and remove flat KV files. The flip side is now that the clone and remove code paths need special casing to handle vvol volumes. Will post the updated PR for review tomorrow. This is not very clean but there are just no APIs to read/write platform KV files at this point.