bmoscon / cryptofeed

Cryptocurrency Exchange Websocket Data Feed Handler
Other
2.19k stars 679 forks source link

Redis Sentinel support #316

Open mfw78 opened 3 years ago

mfw78 commented 3 years ago

Request support for redis sentinel to allow for graceful clustered / failover solutions.

bmoscon commented 3 years ago

Can you give me more info here - I'm not familiar with sentinel. I can see that it exists in the redis libraries, but its unclear to me what you need. Is it a replacement for the normal redis connector? Is it compatible with non-sentinel enabled clusters (or will there need to be support for both modes)?

mfw78 commented 3 years ago

No problems, thanks for the prompt reply @bmoscon.

From my understanding, Sentinel provides High Availability to redis clusters, allowing for a central point that clients may query to get the address of the current cluster's master (essentially, Sentinel acts as an "authority" for which redis instance is master in the cluster, and will work to promote instances in the cluster to master if there is a failure of the current master). Refer https://redis.io/topics/sentinel

I've managed to hack together some brief, functional Sentinel support in cryptofeed / cryptostore, however it currently lacks user configuration, and graceful re-discovery of the master in the event of a master failure. For example, refer attached patch redis.py.txt

To finish this off, the following would be required: