etcd service watcher

[tomfitzhenry]

https://github.com/coreos/etcd is "a highly-available key value store for shared configuration and service discovery", similar to zookeeper.

etcd has a REST API, and there are already Ruby libraries ( https://github.com/ranjib/etcd-ruby , https://github.com/iconara/etcd-rb ) to varying levels of maturity/completeness.

[igor47]

hi! i would definitely be interested in having etcd watchers for synapse as well as etcd publisher in nerve. the nerve part would require some refactoring since we only support a single type of reporter ATM. let me know how i can help!

[bketelsen]

+1

[SEJeff]

https://github.com/ranjib/etcd-ruby supports the v2 etcd protocol. I too would be interested in this.

[arohter]

+1

[cromulus]

+1

[cromulus]

+1

[ChristianBusch]

I am interested in this feature as well.

[calebbrown]

+1

I see PR #58 implements this.

[Hades32]

https://github.com/GoToAssist/synapse/blob/master/synapse/lib/synapse/service_watcher/etcd.rb seems to be the most mature fork AFAICS

[jolynch]

I'm also working with @bobtfish to get this merged into Yelp's synapse fork over at https://github.com/Yelp/synapse/pull/3 and we're going to be merging that back into upstream once we get some more production experience with it.

[jolynch]

As a status update, I'm working on merging #58 with @bobtfish, but we've run into some fun issues with etcd 2.0 version incompatibility and how etcd seems to not implement proper ephemeral nodes ...

I think that even if we did merge the etcd reporter I'm concerned about scalability until they have proper ephemeral nodes. I'm worried smartstack can't scale too far on etcd since you have O(#servers * #services) writes and O(#servers * #services * a lot) reads compared to zookeepers O(#changes) writes and O(#changes) reads from zookeeper. We'll probably end up merging it, but I probably wouldn't recommend it in a deployment of more than 1k servers.

[bitglue]

etcd ephemeral nodes: https://github.com/coreos/etcd/issues/1232