search

kube-object-storage / lib-bucket-provisioner

Library for the dynamic provisioning of object store buckets to be used by object store providers.
Apache License 2.0
20 stars 22 forks source link

Support multiple endpoints #180

Open jeffvance opened 4 years ago

jeffvance commented 4 years ago

It should be possible for a single app pod to consume multiple buckets from the same object store. A pod can mount multiple files (or block devices). If we move the lib to CSI then maybe we get this for free. If not, we should try to design a similar ability for buckets.

For greenfield, perhaps the storage class needs a count parameters key defining the number of new buckets? The associated configMap and secret could use an index suffix to the key-pairs made available to the pod. Or, perhaps we need separate CM and secrets per bucket?

For brownfield, the storage class would need the bucket names and a similar approach for secret and CMs?

copejon commented 4 years ago

Multiple buckets per CM/Secret have some downsides. In cases where 1 developer was not the originator of a bucket, how do they find a desired bucket? If there are more than 1 per CM/Secret, they'd have to examine the key/vals of all CM. And how does a developer scale down the amount of buckets they requested?

On scalability, we had some good feedback at the F2F. Logically, separating sensitive from non-sensitive data makes sense. But at scale, we're doubling the amount of API objects to manage. It may be worth considering merging the CM keys into the Secret's.

That conversation can't happen until we discuss versioning.

jeffvance commented 4 years ago

It may be worth considering merging the CM keys into the Secret's.

See #183