embercsi / ember-csi

Multi-vendor CSI plugin supporting over 80 storage drivers
Other
65 stars 27 forks source link

Volume won't attach because iqn is in an ACL that already exists #154

Open csallen1204 opened 4 years ago

csallen1204 commented 4 years ago

We are using the HPE Left Hand driver and Ember will usually create an ACL on the SAN before mounting to a Kubernetes node to provide access. But it an ACL already exists in the SAN, we'll get this error:

Warning FailedAttachVolume 7s (x6 over 23s) attachdetach-controller AttachVolume.Attach failed for volume "pvc-cd070b5c-e658-45db-92db-2a1dfdfeded6" : rpc error: code = Unknown desc = Exception calling application: Error (HTTP 500) OPERATION_FAILED - The operation failed: The server 'kube17.foo.com' cannot be created because Initiator Node Name 'iqn.2019-12.com.foo.kube17:8692d516cc3' is already in server 'kube17'.

It says it's a warning, but the mount attempt comes to a halt after this happens. Would it be possible to use the existing ACL if the create attempt failed?

csallen1204 commented 4 years ago

Clarification: This seems to only happen when there is another ACL that has the same iqn but a different name i.e. the hostname instead of the fqdn

Akrog commented 4 years ago

I think the best solution is to change the "server" name in the LeftHand array from kube17 to kube17.foo.com.

The alternative would be to report a bug in the Cinder project saying what happens and wait and see if someone fixes it (unlikely given the bug) or fix it yourself, then wait until it gets merged in Cinder, and then do a rebuild of Ember-CSI to pick up the new code with the fix.

Looking at the driver code for this we would have to:

Since this try, fail, retry cycle would happen every time we do an attachment we could use an in-memory dictionary to cache the values for following queries.

Would you be able to rename the "server" in the backend to the node's FQDN?