Jedis connection timeOuts not working as expected

[vinay-sangwan]

My jedis configurations : Set jedisClusterNodes = new HashSet(); jedisClusterNodes.add(new HostAndPort("127.0.0.1", 30001)); jedisClusterNodes.add(new HostAndPort("127.0.0.1", 30002)); jedisClusterNodes.add(new HostAndPort("127.0.0.1", 30003));

  GenericObjectPoolConfig jedisPoolConfig = new GenericObjectPoolConfig();
  jedisPoolConfig.setMaxWaitMillis(10);
  JedisClientConfig jedisClientConfig = DefaultJedisClientConfig.builder().timeoutMillis(50).build();
  jedis = new JedisCluster(jedisClusterNodes,jedisClientConfig,2,jedisPoolConfig);

Steps to reproduce : 1)Start application to establish connection with redis cluster with timeouts set at 50ms only 2)Take redis cluster down 3)Perform operation on redis , eg -> jedis.get("name");

Issue Now I am observing that it is always taking around 1sec of time before throwing the following exception (maxAttempts = 2 , here in this config its timing out before calling renewSlotCache() ) redis.clients.jedis.exceptions.JedisClusterOperationException: Cluster retry deadline exceeded.

Testing with Different Configuration Change maxAttempts = 1 in above mentioned configs

Now above GET operation is throwing exception after 4sec (1sec for first try then 3sec(1 sec each for 3 nodes of cluster) for executing this.provider.renewSlotCache() inside handleConnectionProblem() method of ClusterCommandExecutor )

So I wanted to know if we have a way to minimise this time taken by jedis to throw exception ?

Because if in case of failure in redis cluster its impacting my applications response time

Do let me know if you want any more info on this from my side

[sazzad16]

@vinay-sangwan There is a maxTotalRetriesDuration parameter in JedisCluster. Could you try using that?

Also, which Jedis version are you using?

[vinay-sangwan]

@sazzad16 i am using jedis 4.0.1

Yes i had tried setting maxTotalRetriesDuration by constructor also , that also takes around 1sec Duration maxTotalRetriesDuration = Duration.ofMillis(10); jedis = new JedisCluster(jedisClusterNodes,jedisClientConfig,2,maxTotalRetriesDuration);

Either ways I think we if we dont set maxTotalRetriesDuration , internally it will be set as maxAttempts * timeOuts

[AbhirajRTBA]

how did you resolve this issue i am facing similar issue redis.clients.jedis.exceptions.JedisClusterOperationException: Cluster retry deadline exceeded.

[vinay-sangwan]

@AbhirajRTBA is it happening every time for you , please share configurations as well?

[vimal7225s]

Hi vinay-sangwan, How did you fix this issue. I am facing the same error.

Error was: Cluster retry deadline exceeded. org.mule.runtime.api.connection.ConnectionException: Cluster retry deadline exceeded. Caused by: redis.clients.jedis.exceptions.JedisClusterOperationException: Cluster retry deadline exceeded. at redis.clients.jedis.executors.ClusterCommandExecutor.executeCommand(ClusterCommandExecutor.java:124) ~[jedis-5.1.1.jar:?]

Here is my code

    boolean isSSL = this.tlsContext != null;
    if (isSSL) {
        try {
            SSLContext sslContext = this.tlsContext.createSslContext();
            this.sslParameters = sslContext.getDefaultSSLParameters();
            this.sslParameters.setEndpointIdentificationAlgorithm(
                    this.endpointIdentificationAlgorithm.name().toLowerCase().replace("disabled", ""));
            this.sslParameters.setCipherSuites(this.tlsContext.getEnabledCipherSuites());
            this.sslParameters.setProtocols(this.tlsContext.getEnabledProtocols());

            this.sslSocketFactory = sslContext.getSocketFactory();
        } catch (Exception e) {
            logger.error("Invalid TLSConfiguration Exception");
        }
    }

    JedisClientConfig clientConfig = DefaultJedisClientConfig.builder()
            .ssl(isSSL).sslSocketFactory(sslSocketFactory)
            .user("default")
            .password(connectionParams.getPassword())
            .connectionTimeoutMillis(connectionParams.getConnectionTimeout())
            .hostnameVerifier(null)
            .build();

    Set<HostAndPort> nodes = new HashSet<HostAndPort>();
    nodes.add(new HostAndPort(connectionParams.getHost(), connectionParams.getPort()));

    GenericObjectPoolConfig<Connection> poolConfig = new GenericObjectPoolConfig<Connection>();
    poolConfig.setMaxTotal(10000);
    poolConfig.setMaxIdle(500);

    JedisCluster jedisCluster = new JedisCluster(nodes, clientConfig, poolConfig);

[vinay-sangwan]

@vimal7225s , Please share some more info for me to comment in a better way. Like share timeouts configs , at client side after how much time your call is actually being interrupted and request timed out (you can get time from logs also)

At that time what i observed was that jedis was not accurately honouring command timeouts so i moved that call behind hystrix in thread isolation mode , so now hytrix is able to timeout that call at the desired time.