Create working setup for Kubernetes

mosuka / cete

Cete is a distributed key value store server written in Go built on top of BadgerDB.

Apache License 2.0

234 stars 28 forks source link

Create working setup for Kubernetes #40

Open vniche opened 4 years ago

vniche commented 4 years ago

I will start soon working again on a working cete cluster on Kubernetes and was wondering if anyone here already achieved something of a sort 🤔

mosuka commented 4 years ago

I'd like to run a Cete cluster on Kubernetes, but I haven't started anything yet.

xaocon commented 4 years ago

Just started looking at the project. Sounds pretty neat. I have some helm experience and could make something that would make deployment pretty easy but I don't really understand the requirements yet. What state information do nodes need to join a cluster? Seems like maybe you just need one up and then the rest get --peer-grpc-address with the address of the "leader"? Everything else looks like it's pretty standard across each node except for some concept of ID which I think just needs to be different for each node in the cluster?

vniche commented 4 years ago

I've tried a few setups myself @xaocon, i guess the current implementation does not provide a way to let raft itself elect the leader but only to use a previously chosen one (as you mentioned with --peer-grpc-address), for a initial version you could try working with an OrderedReady StatefulSet, pointing to the first node as the leader.

Maybe we could fetch a few ideas from:

https://github.com/etcd-io/etcd/tree/master/contrib/raftexample
https://medium.com/@kjanshair_4991/using-etcd-raftexample-for-go-b469aad2eadc and reach a dynamic consensus, wether the first healthy node is 0, 1, 2, etc...