search

whatyouhide / redix

Fast, pipelined, resilient Redis driver for Elixir. 🛍
http://hexdocs.pm/redix
MIT License
1.1k stars 121 forks source link

Integrate supporting for Redis Cluster? #152

Closed atlas-comstock closed 4 years ago

atlas-comstock commented 4 years ago

I would like to work on this.

whatyouhide commented 4 years ago

@YongHaoWu before starting work on this, I would really appreciate a detailed proposal on:

  1. Why this would need to be in Redix and not shipped as a separate library.
  2. What changes would be needed in Redix.
  3. What the API would look like.

Thanks for your contribution! 💟

atlas-comstock commented 4 years ago

@whatyouhide Thanks for you detailed reply. : P I would write a proposal a few days later.

PS, I am reading the code of redix, it's so coollll! The blog about state helps a lot. I am writing a blog about analysing code of Redix ;P

Here is the state machine of Redix connection: image

Are the states correct?

whatyouhide commented 4 years ago

Yep, that looks correct :)

atlas-comstock commented 4 years ago

@whatyouhide :P I had another question on https://github.com/whatyouhide/redix/issues/153

whatyouhide commented 4 years ago

Hey @YongHaoWu, any updates on this?

atlas-comstock commented 4 years ago

@whatyouhide Hi, sry for the late reply. I just worked in a new country and quite a lot of things. I have free time from now on. Will give the proposal before 2020😃

whatyouhide commented 4 years ago

Okay @YongHaoWu, thanks for the update. 🙃

atlas-comstock commented 4 years ago

@whatyouhide

Why this would need to be in Redix and not shipped as a separate library.

I think that redis/redis cluster are both Redis thing, and Pipeline, Command functions both have much in common.

Integrating cluster to Redix can make more star/usage, and can be more stable.

Though https://github.com/zhongwencool/redix-cluster wrap a simple Cluster support, it does not support keys on different slots, which means it can not be worked/used in product.

If I were the user of Redis Elixir library, I would not like to use a library that seems lack of support and maintenance.

What changes would be needed in Redix.

How about add a boolen is_cluster parameter to Redix creation?

What the API would look like.

Almost like before. We hide the details for users. They just select the right boolean is_cluster when init Redix.

Like:

{:ok, conn} = Redix.start_link(host: "example.com", port: 5000, is_cluster: true)

How do you think?

whatyouhide commented 4 years ago

Integrating cluster to Redix can make more star/usage

Star and usage are not metrics I am interested in. I am interested in building a reliable and maintainable library, and that is the reason why I'm concerned about adding more functionality to get right and to maintain to Redix :).

How about add a boolen is_cluster parameter to Redix creation?

The API to use Redis Cluster from Redix is only part of the concern here. My main question is: what changes would be needed in the Redix internals and architecture to support Redis Cluster? I have zero experience with Redis Cluster and currently don't have bandwidth to do this research myself 😃

atlas-comstock commented 4 years ago

@whatyouhide Redis Cluster use Hash Slot to distribute data to different nodes, so from driver's perspective, it just need to calculate a query key belongs to which hash slot and do the query to that node.

So the changes needed in the Redix internals and architecture to support Redis Cluster are:

whatyouhide commented 4 years ago

Support multiple Reids server(node) connection

This is not a trivial change. I think one Redis process should stay like this and be a single connection, and not hold multiple connections. I think this essentially needs a pool in front that can route to the right connection based on the key. Does hashing need state? If not, we can build this pool on top of Registry.

whatyouhide commented 4 years ago

Ping @YongHaoWu, has there been any progress on this?

whatyouhide commented 4 years ago

Closing this issue for inactivity. We can reopen this if the discussion will move forward but for now it seems like there is not a huge need for support for Redis Cluster. Thanks everyone!

kennethjeremyau commented 4 years ago

Hi, I started following this issue a few days ago. Having this feature would save me from rolling my own sharding solution, and I think most other clients from other languages have it. Thanks.

whatyouhide commented 4 years ago

@kennethjeremyau thanks for the input! I agree it would nice to have this but currently 1. you can build it on top of Redix itself and 2. I don't have time to work on it, so for now it's not going to happen unless someone wants to contribute it. I'll possibly reopen this issue in case I'll have time to dive into this in the future. Thanks! 💟

alexandremcosta commented 3 months ago

@kennethjeremyau you can use this https://hexdocs.pm/eredis_cluster/ We use in our Elixir app in production for years, and Ericsson too.