redis / redis-py

Redis Python client
MIT License
12.45k stars 2.48k forks source link

Cluster mode connection timeout when using sshtunnel #2800

Closed fidsamurai-crelio closed 2 weeks ago

fidsamurai-crelio commented 1 year ago

Version: redis-py == 4.5.5 redis == 6.0.6

Platform: Python == 3.10.1 Mac (M1) == Monterey

Description: We want to create a script to delete certain Redis keys as they're ephemeral. To allow us to run this from 1 central location (AWS Lambda) we're trying via ssh tunnel. When we try to create the cluster object however we get the connection timeout error. If we connect on a single node however, it works.

Stack Trace

firdausaga@MacBook-Air cacheDelete % python3 cacheDelete.py beta
Region is:  beta
PONG
7000
7001
7002
Traceback (most recent call last):
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/connection.py", line 699, in connect
    sock = self.retry.call_with_retry(
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/retry.py", line 51, in call_with_retry
    raise error
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/connection.py", line 700, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/connection.py", line 1002, in _connect
    raise err
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/connection.py", line 990, in _connect
    sock.connect(socket_address)
TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/cluster.py", line 1491, in initialize
    if bool(r.info().get("cluster_enabled")) is False:
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/commands/core.py", line 991, in info
    return self.execute_command("INFO", **kwargs)
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/client.py", line 1266, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/connection.py", line 1457, in get_connection
    connection.connect()
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/connection.py", line 703, in connect
    raise TimeoutError("Timeout connecting to server")
redis.exceptions.TimeoutError: Timeout connecting to server

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/firdausaga/Documents/CrelioInfra/repos/infra-code/cacheDelete/cacheDelete.py", line 48, in <module>
    redis = RedisCluster(
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/cluster.py", line 607, in __init__
    self.nodes_manager = NodesManager(
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/cluster.py", line 1307, in __init__
    self.initialize()
  File "/Users/firdausaga/Library/Python/3.10/lib/python/site-packages/redis/cluster.py", line 1572, in initialize
    raise RedisClusterException(
redis.exceptions.RedisClusterException: Redis Cluster cannot be connected. Please provide at least one reachable node: Timeout connecting to server

Code

nodes = [ClusterNode(Private, '7000'), ClusterNode(Private, '7001'), ClusterNode(Private, '7002')]
with sshtunnel.open_tunnel(
  SSH,
  ssh_username="ubuntu",
  ssh_pkey=ssh_key,
  remote_bind_addresses=tunnels,
  local_bind_addresses=localPorts
) as tunnel:

  import os

  os.system("redis-cli -h 127.0.0.1 -p 7000 ping")
  for i in localPorts:
    print(i[1])
    with open(str(i[1])+'.txt', 'w') as file:
      r = Redis(host=i[0], port=i[1],decode_responses=True)
      file.write('\n'.join(r.execute_command('keys', 'rq:job:*')))

  from classes import RQJobs
  redis = RedisCluster(
        startup_nodes=nodes,
        decode_responses=False,
        skip_full_coverage_check=True,
        dynamic_startup_nodes=False
  )
  redis.get_nodes()
  #rqDelete = RQJobs(redis, str(i[1])+'.txt')
tunnel.close()
github-actions[bot] commented 1 month ago

This issue is marked stale. It will be closed in 30 days if it is not updated.