search

ryanlecompte / redis_failover

redis_failover is a ZooKeeper-based automatic master/slave failover solution for Ruby.
http://github.com/ryanlecompte/redis_failover
MIT License
539 stars 65 forks source link

Full Failure of server #40

Closed ScotterC closed 12 years ago

ScotterC commented 12 years ago

For the situation where zookeeper, redis and a node manager are on the same server, RedisFailover can't seem to reconcile the situation of losing an entire cluster of these three.

For example:

If I have two servers, each with an instance of redis, zookeeper and a redis_node manager with one in slave to the other and I completely kill the server with the master redis instance, the other server can't reconcile it because - I believe - it can't communicate with those zookeeper instances anymore. If zookeeper can't be communicated with anymore, isn't the best course of action to try and mark that redis instance as unavailable and switch the slave over to master?

ryanlecompte commented 12 years ago

If you only have one ZK server up, you don't have quorum and failover isn't safe. It's recommended to have 5 nodes in your cluster.

On Nov 3, 2012, at 12:42 PM, Scott Carleton notifications@github.com wrote:

For the situation where zookeeper, redis and a node manager are on the same server, RedisFailover can't seem to reconcile the situation of losing an entire cluster of these three.

For example:

If I have two servers, each with an instance of redis, zookeeper and a redis_node manager with one in slave to the other and I completely kill the server with the master redis instance, the other server can't reconcile it because - I believe - it can't communicate with those zookeeper instances anymore. If zookeeper can't be communicated with anymore, isn't the best course of action to try and mark that redis instance as unavailable and switch the slave over to master?

— Reply to this email directly or view it on GitHub.