search

twitter / twemproxy

A fast, light-weight proxy for memcached and redis
Apache License 2.0
12.13k stars 2.06k forks source link

redis HA Automatic Failover #93

Open sdarwin opened 11 years ago

sdarwin commented 11 years ago

From Antirez:

"twemproxy is already able to monitor instance errors, count the number of errors, and eject the node when enough errors are detected. Well it is a shame it is not able to take slave nodes as alternatives, and instead of eject nodes use the alternate nodes just after sending a SLAVE OF NOONE command. This would turn it into an HA solution as well."

Just for a little context, consider ideas from haredis, https://github.com/carlos8f/haredis:

"Easily build a cluster out of 3 or more (default-configured) redis servers Auto-failover due to connection drops Master conflict resolution (default your servers to master, and haredis will elect the freshest and issue the SLAVEOF commands) Freshness judged by an opcounter (incremented on write)"

Proposed solution:

twemproxy monitors a group of shards , each one composed of a group of master-slave nodes. If it's going to eject a node (which it's already capably of), then at this time, it issues SLAVE OF NOONE to a slave, and makes that slave active in the list.

The result of all this is somewhat similar to mongodb with sharding + replicasets, at least in the final outcome.

zuperglue commented 11 years ago

+1

sheldoan commented 11 years ago

+1

javenfang commented 11 years ago

+1

kishore25kumar commented 11 years ago

+1

yankov commented 11 years ago

+1

tovbinm commented 11 years ago

+1 + add support for Sentinel

nettedfish commented 10 years ago

+1 we need it.

bitthegeek commented 10 years ago

+1

manjuraj commented 10 years ago

This will help -- https://github.com/Stono/redis-twemproxy-agent

idning commented 10 years ago

This will help also : https://github.com/idning/redis-mgr

manjuraj commented 10 years ago

@idning redis-mgr looks super cool

idning commented 10 years ago

thanks, we are keep imporving :)

andygrunwald commented 9 years ago

+1

therealbill commented 9 years ago

You can easily use sentinel and if you want something to manage it at a higher level see http://redskull.io

rhoml commented 9 years ago

We solved this issue directly with Redis Sentinel via its reconfigure script. This way we avoid adding more agents or daemons to the infratructure.

andygrunwald commented 9 years ago

@rhoml this is a nice idea. Is this script public?

rhoml commented 9 years ago

@andygrunwald yes, we have this version https://gist.github.com/rhoml/fecb88ce4500ef22b05f. You'll see 2 twemproxy's because we are sharding redis and we are using 2 twemproxy's for making the process easier in AWS when creating new shards.

andygrunwald commented 9 years ago

Great. Thanks @rhoml!