asyncio RedisCluster is not working for Amazon Elasticache

[mvbrn]

Version: Redis-py redis==4.3.3 (hiredis==2.0.0)

Redis 6.0.5 on Amazon Elasticache

Platform:

Python 3.9.11 on Debian GNU/Linux 11 (bullseye)

Description:

asyncio RedisCluster client is hanging when trying to be connected to Redis Cluster hosted on Amazon Elasticache. No exception raised, it's just freezing on initialize() forever. At the same time, using blocking RedisCluster for the same cluster with the same parameters is working fine. Single-node asyncio Redis client is also working when connecting directly to a cluster node.

A small snippet of how to reproduce:

import asyncio
from redis.asyncio.cluster import RedisCluster as AsyncRedisCluster

redis_cluster = AsyncRedisCluster.from_url(
    "rediss://:REDACTED@clustercfg.REDACTED.REDACTED.REDACTED.cache.amazonaws.com:6379",
    decode_responses=True,
    require_full_coverage=False
)

async def go():
    await redis_cluster.initialize()
    await redis_cluster.ping()

asyncio.run(go())

[utkarshgupta137]

Looks like you're using SSL, which is currently broken in async cluster. I've already raised a PR.

Fixed by: https://github.com/redis/redis-py/pull/2217

[wckdman]

I faced the same problem on my on-premise Redis cluster with password auth:

• works fine with blocking RedisCluster
• fails to connect via async RedisCluster: RedisClusterException: Redis Cluster cannot be connected. Please provide at least one reachable node.

Version: redis==4.3.3 (tried pip and source from your branch)

Python 3.9.11

[utkarshgupta137]

Can you share the command you're using to create RedisCluster? Can you also try with the changes in #2217 & share the full traceback?

[wckdman]

Here's the commands. I tried them with changes in https://github.com/redis/redis-py/pull/2217

from redis.asyncio.cluster import ClusterNode as AsyncClusterNode, RedisCluster as AsyncRedisCluster
from redis.cluster import ClusterNode, RedisCluster

redis_nodes = [("host1", 6379), ("host2", 6379)]
startup_nodes = []
startup_nodes_async = []
for host, port in redis_nodes:
    startup_nodes.append(ClusterNode(host, port))
    startup_nodes_async.append(AsyncClusterNode(host, port))

redis = RedisCluster(startup_nodes=startup_nodes, password="password")
redis.ping()

>> True

async_redis = AsyncRedisCluster(startup_nodes=startup_nodes_async, password="password")
await async_redis.initialize()

Traceback:

---------------------------------------------------------------------------
AuthenticationError                       Traceback (most recent call last)
File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/cluster.py:1068, in NodesManager.initialize(self)
   1066 try:
   1067     # Make sure cluster mode is enabled on this node
-> 1068     if not (await startup_node.execute_command("INFO")).get(
   1069         "cluster_enabled"
   1070     ):
   1071         raise RedisClusterException(
   1072             "Cluster mode is not enabled on this node"
   1073         )

File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/cluster.py:894, in ClusterNode.execute_command(self, *args, **kwargs)
    893 try:
--> 894     return await self.parse_response(connection, args[0], **kwargs)
    895 finally:
    896     # Release connection

File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/cluster.py:873, in ClusterNode.parse_response(self, connection, command, **kwargs)
    872     else:
--> 873         response = await connection.read_response_without_lock()
    874 except ResponseError:

File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/connection.py:957, in Connection.read_response_without_lock(self, disable_decoding)
    956     else:
--> 957         response = await self._parser.read_response(
    958             disable_decoding=disable_decoding
    959         )
    960 except asyncio.TimeoutError:

File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/connection.py:533, in HiredisParser.read_response(self, disable_decoding)
    532 if isinstance(response, ConnectionError):
--> 533     raise response
    534 elif (
    535     isinstance(response, list)
    536     and response
    537     and isinstance(response[0], ConnectionError)
    538 ):

AuthenticationError: Authentication required.

The above exception was the direct cause of the following exception:

RedisClusterException                     Traceback (most recent call last)
Input In [6], in <cell line: 1>()
----> 1 await async_redis.initialize()

File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/cluster.py:339, in RedisCluster.initialize(self)
    337 self._initialize = False
    338 try:
--> 339     await self.nodes_manager.initialize()
    340     await self.commands_parser.initialize(
    341         self.nodes_manager.default_node
    342     )
    343 except BaseException:

File ~/.pyenv/versions/miniconda3-latest/envs/redis-cluster/lib/python3.9/site-packages/redis-4.3.3-py3.9.egg/redis/asyncio/cluster.py:1175, in NodesManager.initialize(self)
   1172         break
   1174 if not startup_nodes_reachable:
-> 1175     raise RedisClusterException(
   1176         "Redis Cluster cannot be connected. Please provide at least "
   1177         "one reachable node. "
   1178     ) from exception
   1180 # Check if the slots are not fully covered
   1181 if not fully_covered and self.require_full_coverage:
   1182     # Despite the requirement that the slots be covered, there
   1183     # isn't a full coverage

RedisClusterException: Redis Cluster cannot be connected. Please provide at least one reachable node. 

[utkarshgupta137]

Thanks! I've updated the PR, should be fixed now.

[wckdman]

worked for me, Thank you!

I'm looking forward to new release :)

[rhoboro]

@utkarshgupta137 I faced the same problem with Amanzon MemoryDB for Redis. And https://github.com/redis/redis-py/pull/2217 works fine. Thank you!

It would be better if there is support for rediss:// schema as well as ssl parameter.

[utkarshgupta137]

@utkarshgupta137 I faced the same problem with Amanzon MemoryDB for Redis. And #2217 works fine. Thank you!

It would be better if there is support for rediss:// schema as well as ssl parameter.

RedisCluster.from_url should work with SSL as well.

[rhoboro]

Thank you for the comment. In my case, RedisCluster.from_url("redis://...", ..., ssl=True) works but RedisCluster.from_url("rediss://...", ...) does not work. I expected that rediss:// shema works like ssl=True.

[utkarshgupta137]

Thank you for the comment. In my case, RedisCluster.from_url("redis://...", ..., ssl=True) works but RedisCluster.from_url("rediss://...", ...) does not work. I expected that rediss:// shema works like ssl=True.

Updated the PR with a fix.

[dvora-h]

fixed in #2217

[wckdman]

@dvora-h Hello Could you create a minor release with this bugfix please ?:)

[dvora-h]

This is included in version 4.4.0rc1 I hope a stable version will be released soon