search

turms-im / turms

🕊️ The world's most advanced open source instant messaging engine for 100K~10M concurrent users https://turms-im.github.io/docs
Apache License 2.0
1.76k stars 275 forks source link

redis cluster support #1517

Open zhaieryuan opened 3 months ago

zhaieryuan commented 3 months ago

请问改造redis 为 cluster模式该怎么处理呢？希望作者能提供一下，谢谢。

JamesChenX commented 3 months ago

Currently, not all modules (e.g. blocking users/IPs) of servers support working with Redis cluster as some Lua scripts provided by servers are designed to work for standalone Redis instances efficiently, but cannot work with the Redis cluster.

I plan to add Redis cluster support soon and will try to publish it next weekend. I will @ you once the support is published.

zhaieryuan commented 3 weeks ago

请问作者，redis cluster 这里改造的怎么样了，我这边看了官方的api，但是这里涉及到关于codec的一些功能，所以改造起来耗时了很长时间，也是不能很好的解决，请作者这里帮忙提供一些改造的思路，或者是伪代码，感谢支持。

JamesChenX commented 3 weeks ago

Sorry about it. The task is deferred after research and there is no plan to support it in recent months (I am busy with turms-chat-demo currently, a complete demo client).

I found that supporting Turms in running with the Redis cluster should NOT use exactly the same logic and data in the standalone mode and cluster mode, which will take a great effort because:

  1. When running in standalone mode, we design the data in Redis in an efficient way: We try to store data in a standalone Redis so that we can handle all related data in one request/script.
  2. But when running in cluster mode, we should design the data in Redis in a distributed way to use the Redis cluster in a more robust way.

As they (the standalone and the cluster) are designed for different purposes and solving different problems, we need to use different logic and data in Redis, the following tasks should be addressed to support running with the Redis cluster:

  1. Support im.turms.server.common.storage.redis.TurmsRedisClient in running with the Redis cluster. For example: a. choosing different Redis instances to handle the Redis requests if the instances/topology of the Redis cluster have changed. This should be easy if you know how the Redis cluster works. b. support using both keys and args in Redis scripts, as we only use keys currently. This should be easy too.
  2. Design efficient logics and data to store data for different Turms modules (e.g. data for blocking users; data for online user sessions) in the Redis cluster. This take will take a great effort. You can choose the one you use to implement so that you don't have to support the Redis cluster for all modules at the first stage.
  3. (Optional) Support migrating data from the standalone mode to the cluster mode.

For most applications, using one Redis standalone should be enough for performance, but you will get a big problem if the Redis instance crashes for any reason (btw, I did encounter a machine crash physically). So you can either wait for me to support it after I finish current tasks, or support it by yourselves in the above way or your (maybe incompatible) way.