Cannot connect to elasticache instance from the start

[bluermind]

I've set up my Redis cluster. I can't go past simply the connection stage

from rediscluster import RedisCluster
startup_nodes = [{ "host": "xxxxxx.xxx.clustercfg.apn2.cache.amazonaws.com", "port": "6379" }]
rc = RedisCluster(startup_nodes=startup_nodes, ssl=True, ssl_cert_reqs=None, decode_responses=True, skip_full_coverage_check=True)

I've checked that I am able to connect to the cluster using RedisInsight client. Any guesses? Your help would be appreciated Python just stays silent without any traces..

[Grokzen]

@bluermind I don't know really. You need to provide more information then this really. What command you run when it happens? Have you followed the example that exists here? https://github.com/Grokzen/redis-py-cluster/blob/master/examples/basic_elasticache_password_protected.py maybe omit the password if you do not run that. When you press ctrl+c to exit out from your silent script run, what stack trace you get then? aka where do your code get stuck.

Without any of this i can't help any further. Also in general, random questions about the usage of this lib or other general questions about usage from elasticache should be asked over at stackoverflow and not here. This place is more for when you have traced down errors and issues and not so much general questions. But if you provide more debug and logs and what is happening i can help further.

[bluermind]

@Grokzen Thanks a lot for your reply. I am sorry that I couldn't provide further info. My code above follows just about everything that the link guides, without the password. My ctrl+c (keyboard interrupt) didn't execute for this occasion. (My other python apps take in keyboard interrupt, just not this case) So I could not provide any tracebacks since I had to kill the python process by force.

I will try connecting using Java and try to confirm that it's not due to set up in Elasticache.

Edit: I tried my keyboard interrupt again and now it prints the traces.

Traceback (most recent call last):
  File "redis_operation.py", line 6, in <module>
    rc = RedisCluster(startup_nodes=startup_nodes, ssl=True, ssl_cert_reqs=None, decode_responses=True, skip_full_coverage_check=True)
  File "/home/bluermind/.local/lib/python3.6/site-packages/rediscluster/client.py", line 375, in __init__
    **kwargs
  File "/home/bluermind/.local/lib/python3.6/site-packages/rediscluster/connection.py", line 160, in __init__
    self.nodes.initialize()
  File "/home/bluermind/.local/lib/python3.6/site-packages/rediscluster/nodemanager.py", line 226, in initialize
    cluster_slots = r.execute_command("cluster", "slots")
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/client.py", line 898, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/connection.py", line 1192, in get_connection
    connection.connect()
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/connection.py", line 559, in connect
    sock = self._connect()
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/connection.py", line 861, in _connect
    sock = ssl_wrap_socket(context, sock, server_hostname=self.host)
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/_compat.py", line 17, in ssl_wrap_socket
    return context.wrap_socket(sock, *args, **kwargs)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
KeyboardInterrupt
Exception ignored in: <object repr() failed>
Traceback (most recent call last):
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/client.py", line 885, in __del__
    self.close()
  File "/home/bluermind/.local/lib/python3.6/site-packages/redis/client.py", line 888, in close
    conn = self.connection
AttributeError: 'RedisCluster' object has no attribute 'connection'

[bluermind]

Updates: I tried connecting to Elasticache cluster from within the same AWS vpc, without any luck. Also, I did telnet redis_endpoint 6379 and confirmed that I had connectivity to the cluster from my local machine.

[Grokzen]

If i remember correctly, there is two endpoints that you have in the elasticcache where one of them will just return a data structure that points you to all nodes, and you have one url for each node in the cluster and the first one works and the second one do not. What this lib requires is a URL endpoint to your redis nodes inside your cluster, not the config helper url endpoint that AWS provides.

[bluermind]

Holy.. is that so. Then I guess we come down to a configuration decision. I think there are three modes that you can set up Redis in Elasticache. First is standalone where we have one endpoint for both read and write, in which case we will just use the redis-py package. Second and third are cluster modes, whereby the former is labelled as (the labeling may lead to some confusion here) 'cluster disabled' and latter as 'cluster enabled'. In the cluster disabled mode, it is configured to have master-slave nodes where you have a primary endpoint and a reader endpoint. You will write your records to this endpoint which will be replicated over the slave nodes and then read from the reader endpoint. Again for such cases, I could set up two clients, one for write and one for read using the redis lib. Now the cluster enabled mode stores the records across different nodes inside the cluster. And the configuration endpoint controls where data goes and from which node I read the keys from. If I have to manage each connection to every node in that cluster and keep a record of which slots every data belongs to, and if the lib cannot somehow manage this on its own, it will cause a huge pain. I haven't begun to explore this lib, since I couldn't get past the connection stage, so I might be having wrong understanding of this library's capabilities. So please correct me if I am wrong, and let me know if I was to fallback to the cluster mode disabled option (the second option from above), would it make sense to use this lib or would we be better to create two connections using the redis-py library?

[Grokzen]

@bluermind I don't know really. If you want a cluster then use this lib. If you want a standalone instance and/or with sentinel then use redis-py. From your initial issue you reported, all you need is either an IP or a DNS name to one node inside your cluster for this lib to work, nothing more or less really. Elasticache confuses people with having two endpoints, one is a convenience endpoint and the other what we really want to use here.