search

NetApp / netappdvp

A Docker volume plugin for NetApp storage
96 stars 33 forks source link

Calls issued through Docker Volume API seem to occasionally be *lost* #59

Closed j-griffith closed 7 years ago

j-griffith commented 7 years ago

I don't think this is an NDVP issue, but while doing some testing lately I've noticed occasionally I'll run into scenarios where I issue a "docker volume create -d netapp xxxx" call, and the call is never received at the ndvp daemon. Docker goes about life happily, and add the volume to it's data-base with the netapp driver being the owner, but the volume doesn't actually exist anywhere.

I think the Docker Volume API is kinda garbage by interpreting "no" response as a good response, we should consider a PR to docker that uses an ack (like a 202 response or something) on calls to verify that they actually made it to the destination and were succesful rather than just relying on an error being returned.

j-griffith commented 7 years ago

Checked out the docker logs, and it seems we occasionally see this:

Jan 13 16:47:49 clabb10603 dockerd[1678]: time="2017-01-13T16:47:49.622276503-07:00" level=error msg="Handler for POST /v1.24/volumes/create returned error: create clone_1: VolumeDriver.Create: Error creating storage: Failed to find source volume: error: Failed to retrieve volume by name in snapshot list operation; name: volume_t Jan 14 10:57:54 clabb10603 dockerd[1678]: time="2017-01-14T10:57:54.874817840-07:00" level=warning msg="Unable to connect to plugin: /run/docker/plugins/netapp.sock/VolumeDriver.List: Post http://%2Frun%2Fdocker%2Fplugins%2Fnetapp.sock/VolumeDriver.List: dial unix /run/docker/plugins/netapp.sock: connect: connection refused, retr Jan 14 10:57:55 clabb10603 dockerd[1678]: time="2017-01-14T10:57:55.875334568-07:00" level=warning msg="Unable to connect to plugin: /run/docker/plugins/netapp.sock/VolumeDriver.List: Post http://%2Frun%2Fdocker%2Fplugins%2Fnetapp.sock/VolumeDriver.List: dial unix /run/docker/plugins/netapp.sock: connect: connection refused, retr Jan 14 10:57:57 clabb10603 dockerd[1678]: time="2017-01-14T10:57:57.875786815-07:00" level=warning msg="Unable to connect to plugin: /run/docker/plugins/netapp.sock/VolumeDriver.List: Post http://%2Frun%2Fdocker%2Fplugins%2Fnetapp.sock/VolumeDriver.List: dial unix /run/docker/plugins/netapp.sock: connect: connection refused, retr Jan 14 10:58:01 clabb10603 dockerd[1678]: time="2017-01-14T10:58:01.876311213-07:00" level=warning msg="Unable to connect to plugin: /run/docker/plugins/netapp.sock/VolumeDriver.List: Post http://%2Frun%2Fdocker%2Fplugins%2Fnetapp.sock/VolumeDriver.List: dial unix /run/docker/plugins/netapp.sock: connect: connection refused, retr

Need to spend some more time digging into this, but I think the Docker service should catch this.

adkerr commented 7 years ago

Internally tracked by [DE312]

korenaren commented 7 years ago

@j-griffith you okay to close this as unreproducible?

j-griffith commented 7 years ago

Yep, I'll close it now since there don't seem to be any objections.