search

kube-object-storage / lib-bucket-provisioner

Library for the dynamic provisioning of object store buckets to be used by object store providers.
Apache License 2.0
20 stars 22 forks source link

Deleted OBCs are waiting in the provisioning queue #187

Closed romayalon closed 4 years ago

romayalon commented 4 years ago

lib-bucket-provisioner should make sure to remove OBC's from the queue when they are deleted. scenario: when creating a faulty OBC, the provisioning do not succeed, after deleting the OBC, the next correct OBC creation takes a lot of time. Suggested fix: The queue should also contain the UID too so that delete-create will not reuse the same item.

nimrod-becker commented 4 years ago

@copejon @jeffvance Would love to get your input on this We would like to fix this and would love to get your inputs

copejon commented 4 years ago

Will take a look now.

copejon commented 4 years ago

@romayalon What version of the library are you running? I'm having trouble reproducing this on master:HEAD. This was a known issue that was only fixed in the last couple commits.

nimrod-becker commented 4 years ago

@copejon referencing the BZ https://bugzilla.redhat.com/show_bug.cgi?id=1785260#c4

copejon commented 4 years ago

@nimrod-becker Thank you, that should give a better picture of what's going on. I'll read & test and let you know what I find.

copejon commented 4 years ago

@nimrod-becker @romayalon It seems like there are 2 issues described in the BZ:

1 - deleting an erroneous OBC and recreating it causes creation to take an unexpectedly long time. The bug was that obc keys were not being dropped from the queue on deletes. #184 fixes this. Please update the noobaa operator's version of the library.

2 - That OBCs whose storageClasses don't exist are retried on an exponential backoff. This is a desired behavior of k8s controllers - to make a best effort to make Actual State of the World == Desired State of the World. However, the user isn't informed of what's going on, which is a deficiency on our part. #68 needs to be addressed here

romayalon commented 4 years ago

Updated lib-bucket-provisioner to the latest version and now it works as expected. Also i've added errors in NooBaa for the specific bug. Thanks

jeffvance commented 4 years ago

good new @romayalon ! I'll close this issue.