davissp14 / etcdv3-ruby

Etcd v3 Ruby Client
MIT License
52 stars 17 forks source link

Namespacing Support #116

Closed jcalvert closed 3 years ago

jcalvert commented 6 years ago

Namespacing is a common pattern in key/value stores when multiple application usages are present as a strategy to avoid name conflicts and data leakage. Support for this pattern is found with Ruby libraries for Memcached and Redis and the Go language Etcd client.

An implementation of this might look like passing a namespace constructor argument, similar to timeout: Etcdv3.new(:endpoints => “http://foo”, :namespace => “app1:”) - calls to this object would then operate on keys within the namespace with a prefix in a transparent way that does not require the caller to remember to add or remove this prefix.

While investigating how to potentially add support for this feature, we noted that calls return the Protobuf generated objects which have immutable strings. This means that for instance a get request would have a response that would have the prefixed key in the result: <Mvccp:KeyValue: key: “namespace:foo” … - which could lead to errors if this value is read and subsequently used. Since this object cannot be modified to solve this problem we would need to wrap this in a proxy object.

We were hoping to get your thoughts on this approach and issue before digging into this further as this is a pretty significant change even if the interface does not change.

cc/ @mgates

davissp14 commented 6 years ago

I like the general concept.

The Etcd Client implementation is looping through the response data and ripping the prefix out of the result before returning to the user. Not crazy about the implementation, but I think it would be fine as long as that logic is abstracted away from our existing handlers.

https://github.com/coreos/etcd/blob/049ca8746a1d3ab17c4448425074537584fe3006/clientv3/namespace/kv.go

I haven't thought super hard about how we could implement this, but having a separate group of handlers that considers namespace seems like it could be a sane approach.

https://github.com/davissp14/etcdv3-ruby/blob/master/lib/etcdv3/connection.rb#L4-L10

davissp14 commented 6 years ago

Thoughts?

mgates commented 6 years ago

Hey - That approach makes a lot of sense, but I think we have to back off of this for now. Hopefully we'll have some time to pick it up in the future though. Thanks for the guidance.

davissp14 commented 3 years ago

https://github.com/davissp14/etcdv3-ruby/pull/134

davissp14 commented 3 years ago

Merged. Available in 0.11.0.