Grokzen / redis-py-cluster

Python cluster client for the official redis cluster. Redis 3.0+.
https://redis-py-cluster.readthedocs.io/
MIT License
1.1k stars 316 forks source link

Command: WAIT is blocked in redis cluster mode #486

Closed Vabs3 closed 1 year ago

Vabs3 commented 2 years ago

Hi

I am working with a Redis cluster. I have setup 3 masters with 1 replica each. I am using redis-py-cluster as a client. While running wait command to synchronise writes to replicas, I am getting following error:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/concurrent/futures/process.py", line 175, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "test_redis_new_wait.py", line 235, in method_time_calculator
    method(*args)
  File "test_redis_new_wait.py", line 41, in setnx
    do_wait()
  File "test_redis_new_wait.py", line 31, in do_wait
    CLIENT.wait(WAIT, WAIT_TIMEOUT)
  File "/home/vaibhavgrover/redis_benchmarking/venv/lib64/python3.6/site-packages/redis/client.py", line 1490, in wait
    return self.execute_command('WAIT', num_replicas, timeout)
  File "/home/vaibhavgrover/redis_benchmarking/venv/lib64/python3.6/site-packages/rediscluster/client.py", line 555, in execute_command
    return self._execute_command(*args, **kwargs)
  File "/home/vaibhavgrover/redis_benchmarking/venv/lib64/python3.6/site-packages/rediscluster/client.py", line 579, in _execute_command
    node = self.determine_node(*args, **kwargs)
  File "/home/vaibhavgrover/redis_benchmarking/venv/lib64/python3.6/site-packages/rediscluster/client.py", line 527, in determine_node
    return blocked_command(self, command)
  File "/home/vaibhavgrover/redis_benchmarking/venv/lib64/python3.6/site-packages/rediscluster/utils.py", line 46, in blocked_command
    raise RedisClusterException("Command: {0} is blocked in redis cluster mode".format(command))
rediscluster.exceptions.RedisClusterException: Command: WAIT is blocked in redis cluster mode

It is saying wait is blocked in redis cluster mode. But I am able to run wait command in redis-cli -c utility.

Thanks

Grokzen commented 2 years ago

Commands that is not possible to determine what node it should be routed to can't be implemented in a cluster mode w/o some additional capability of selecting a node to send the command to. If that is not implemented, which it is not then you can't use that feature in this lib. If you have any suggestions on how to implement this of a reference in how any other lib have done it then i can implement it.

Vabs3 commented 2 years ago

Thanks for the reply. Got the problem. I will look for reference.

Suggestion:

Can we create wait_set(), wait_setnx() etc. similar methods. Which will basically run wait after set on the same server. May be a pipeline of these two commands. What do you think?

Also is there any other way to make sure the values are copied to replicas?

Grokzen commented 2 years ago

No i will not make a override specific version for every single method that we have, that will not fly. You are coming back to the main issue that by simply telling the redis cluster to "WAIT" you can't know just from that what nodes you want to wait on and for what. You either need a identifier or you need to send it to all nodes but that do not make sense either. Same thing goes for a pipeline really, that do not help.

I do not know how to solve your problem in the current code as there is no cluster node determine functionality within a WAIT command and it depends on the current connection which makes sense in a single node and using Redis class but not a multi node and multi connection environment.

Grokzen commented 1 year ago

Closing due to inactivity. Also if this feature is wanted @Vabs3 you have to now raise it up to the git repo redis-py that now have cluster support to see if they are willing to fix or support this feature. It won't be created or built here at least.