redis / lettuce

Advanced Java Redis client for thread-safe sync, async, and reactive usage. Supports Cluster, Sentinel, Pipelining, and codecs.
https://lettuce.io
MIT License
5.3k stars 947 forks source link

AWS RedisCache replica Failover can not be resolved #2872

Open musicallyvk opened 3 weeks ago

musicallyvk commented 3 weeks ago

Hi Everyone, Hope everyone is doing great!

I know this issue might not related to lettiuce but because it is redis then might be someone would have encountered the same.

so i have a 3 replica and 1 primary on aws elasticache redisCache using version 4.0, with the redis config file on the application level which has a load balancer with round robbin and using replicatedServersConfig. and many other things like retry , failedSlaveCheckInterval,timeout, readMode , subscriptionMode and many things.

now the thing is if a failover happens then some requests are still being routed to that failed node, how to get it resolved, anyone please help me. i am stuck with this for months. Thanks Already !

if you need more details on config then -

{
   "replicatedServersConfig":{
      "idleConnectionTimeout":10000,
      "connectTimeout":10000,
      "timeout":30000,
      "retryAttempts":3,
      "retryInterval":1500,
      "failedSlaveReconnectionInterval":3000,
      "failedSlaveCheckInterval":60000,
      "password":"*****",
      "subscriptionsPerConnection":5,
      "clientName":"myClientName",
      "loadBalancer":{
         "class":"org.redisson.connection.balancer.RoundRobinLoadBalancer"
      },
      "subscriptionConnectionMinimumIdleSize":1,
      "subscriptionConnectionPoolSize":50,
      "slaveConnectionMinimumIdleSize":24,
      "slaveConnectionPoolSize":64,
      "masterConnectionMinimumIdleSize":24,
      "masterConnectionPoolSize":64,
      "readMode":"SLAVE",
      "subscriptionMode":"SLAVE",
      "nodeAddresses":[
         "rediss://Node1,Node2,Node3,Node4"
      ],
      "scanInterval":1000
   },
   "threads":16,
   "nettyThreads":32,
   "codec":{
      "class":"org.redisson.codec.LZ4Codec"
   },
   "transportMode":"NIO"
}
tishun commented 3 days ago

Hey @musicallyvk ,

I assume - since you are using a load balancer - that the Lettuce driver is set up in a way that it only connects to the load balancer? If so this seems very specific to the AWS configuration I doubt you will find much help here. Have you tried asking in a public forum like stackoverflow? Or one that covers AWS?