Grokzen / redis-py-cluster

Python cluster client for the official redis cluster. Redis 3.0+.
https://redis-py-cluster.readthedocs.io/
MIT License
1.09k stars 315 forks source link

Support for transactions and watch function #238

Open alwaysR9 opened 6 years ago

alwaysR9 commented 6 years ago

In Redis cluster tutorial, it says :

Migrating to Redis Cluster

Users willing to migrate to Redis Cluster may have just a single master, or may already using a preexisting sharding setup, where keys are split among N nodes, using some in-house algorithm or a sharding algorithm implemented by their client library or Redis proxy. In both cases it is possible to migrate to Redis Cluster easily, however what is the most important detail is if multiple-keys operations are used by the application, and how. There are three different cases:

  1. Multiple keys operations, or transactions, or Lua scripts involving multiple keys, are not used. Keys are accessed independently (even if accessed via transactions or Lua scripts grouping multiple commands, about the same key, together).
  2. Multiple keys operations, transactions, or Lua scripts involving multiple keys are used but only with keys having the same hash tag, which means that the keys used together all have a {...} sub-string that happens to be identical. For example the following multiple keys operation is defined in the context of the same hash tag: SUNION {user:1000}.foo {user:1000}.bar.
  3. Multiple keys operations, transactions, or Lua scripts involving multiple keys are used with key names not having an explicit, or the same, hash tag.

    The third case is not handled by Redis Cluster: the application requires to be modified in order to don't use multi keys operations or only use them in the context of the same hash tag. Case 1 and 2 are covered, so we'll focus on those two cases, that are handled in the same way, so no distinction will be made in the documentation.

If I understand correctly, the redis cluster supports case 1 and case 2.

But I find this project do not implement multi, watch yet. I think we should support for case 1 and case 2, but do not support for case 3. Any suggestion would be appreciated.

Grokzen commented 6 years ago

@alwaysR9 There is a open PR that might implement watch/multi, but the issue is deeper then that and required a lot of thinking and good code to properly work in a cluster environment.

If i understand the text correct, then i have the following comments.

  1. There is several multi key commands that is already implemented and works. For example MGET. The trick for these methods is that you have to work the multi host problem in the client instead of letting the redis cluster handle it. I would say this is implemented.
  2. Multikey operations that has the same hashslot will work becuase it is not really multiserver but multikey if you point all keys to the same hash slot. The thing here is that transactions is not supported and probably will not work without some downsides and possible consistency problems in a cluster envrinoment. You should really look into the docs for this project becuase i have written extensively on this problem and possible ideas and solutions. But no matter what solution, you have to sacrefice something to gain the feature, but transactions is a real problem becuase you can get into situations where transactions is commited to some servers but not all servers and then would you really have transactional support if your transaction is only partyl applied or partly rolled-back?? These are hard questions and my take on it is that redis do not offer a good solution to this problem of multiserver prep/commit.
  3. This is also possible to do with multikey support. It uses the same solution as 2. It is much easier to implement the features so that it will work in both 2. and 3. The problem is that not all functions is implemented in the same way as a single server and some commands almost can't be implemented because of the complexity of them.