redisson / redisson

Redisson - Easy Redis Java client and Real-Time Data Platform. Valkey compatible. Sync/Async/RxJava/Reactive API. Over 50 Redis or Valkey based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Bloom filter, Spring, Tomcat, Scheduler, JCache API, Hibernate, RPC, local cache...
https://redisson.pro
Apache License 2.0
23.26k stars 5.34k forks source link

Intermittent Exception when creating RedissonLocalCachedMap in 3.24.1 #5385

Closed OS-tariqabbasi closed 11 months ago

OS-tariqabbasi commented 11 months ago

Expected behavior

No errors, same as with 3.23.5

Actual behavior

After upgrading to 3.24.1 my application is ocassionally throwing this error:

org.redisson.client.RedisTimeoutException: Unable to acquire subscription lock after 0ms. Try to increase 'subscriptionTimeout', 'subscriptionsPerConnection', 'subscriptionConnectionPoolSize' parameters.
    at org.redisson.pubsub.PublishSubscribeService.lambda$timeout$12(PublishSubscribeService.java:328)
    at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
    at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
    at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
    at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
    at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:833)

The application uses RedissonLocalCachedMap with SyncStrategy.NONE. It doesn't override the default subscriptionTimeout.

Steps to reproduce or test case

This code reproduces the bug most of the time with 3.24.1, never throws with 3.23.5:

    void redissonSubscribeIssue() {
        Config config = new Config();
        config.useSingleServer()
                .setAddress("redis://localhost:6379")
                .setConnectionMinimumIdleSize(1)
                .setConnectionPoolSize(16)
                .setSubscriptionConnectionMinimumIdleSize(1)
                .setSubscriptionConnectionPoolSize(32)
                .setUsername(null)
                .setPassword("...")
                .setTcpNoDelay(true)
                .setKeepAlive(true)
                .setTimeout(30000)
                .setConnectTimeout(10000)
                .setPingConnectionInterval(10000);
        RedissonClient redissonClient = Redisson.create(config);
        for (int i = 0; i < 8; i++) {
            redissonClient.getLocalCachedMap(
                    UUID.randomUUID().toString(),
                    LocalCachedMapOptions
                            .<String, String>defaults()
                            .syncStrategy(SyncStrategy.NONE)
                            .useKeyEventsPattern(false)
            );
        }
    }

Redis version

Observed with 7.0.9 cluster and single instance.

Redisson version

3.24.1

Redisson configuration

See code block above.

mrniko commented 11 months ago

pubsub timeout was extracted into a separate setting subscriptionTimeout. For your case you need to set it 3*1500+30000=34500

OS-tariqabbasi commented 11 months ago

pubsub timeout was extracted into a separate setting subscriptionTimeout. For your case you need to set it 3*1500+30000=34500

Added .setSubscriptionTimeout(34500) to the test case above and it's still breaking sometimes. Same error but sometimes it has 1ms instead now:

org.redisson.client.RedisTimeoutException: Unable to acquire subscription lock after 1ms. Try to increase 'subscriptionTimeout', 'subscriptionsPerConnection', 'subscriptionConnectionPoolSize' parameters.
mrniko commented 11 months ago

what about 60000?

OS-tariqabbasi commented 11 months ago

Also fails with 60,000 and the test is taking less than 1 second to run. Perhaps there is an incorrect unit conversion somewhere? Increasing the number of created maps makes it fail more often.

OS-tariqabbasi commented 11 months ago

It's failing sometimes even with .setSubscriptionTimeout(Integer.MAX_VALUE) and creating 16 maps, the increased timeout does seem to make this happen much less often with a small number of maps. Can be reproduced reliably with 64 maps. On 3.23.5 this test passes reliably with over 100 maps.

mrniko commented 11 months ago

Fixed. Thanks for report