redis / lettuce

Advanced Java Redis client for thread-safe sync, async, and reactive usage. Supports Cluster, Sentinel, Pipelining, and codecs.
https://lettuce.io
MIT License
5.37k stars 965 forks source link

Why Lettuce leaves lot of threads open even after closing the connection and shutting down the cluster #2046

Closed HarishMalavade closed 2 years ago

HarishMalavade commented 2 years ago

Current Behavior

We are using Lettuce library to perform operations against AWS redis cluster and below is sample snippet, When this is executed under load some of the lettuce threads are not released and still active and most of them are

What do I need to do to close these above threads so my JVM is healthy and releasing the threads and I dont get into out of memory issues.


RedisURI redisUri = RedisURI.Builder
                .redis(ApplicationScopedComponents.CONFIG.getProperties().getProperty("REDIS_URL"))
                .withPort(Integer.valueOf(ApplicationScopedComponents.CONFIG.getProperties().getProperty("REDIS_PORT")))
                .withSsl(Boolean
                        .valueOf(ApplicationScopedComponents.CONFIG.getProperties().getProperty("REDIS_USE_SSL")))
                .withPassword(ApplicationScopedComponents.CONFIG.getProperties().getProperty("REDIS_AUTH"))
                .build();
ClientResources resource = DefaultClientResources.builder().ioThreadPoolSize(10).computationThreadPoolSize(10).build();
                                    ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
                            .closeStaleConnections(true)
                                            .enableAllAdaptiveRefreshTriggers()
                                            .build();
RedisClusterClient redisClusterClient = RedisClusterClient.create(resource, redisUri);
StatefulRedisClusterConnection<String, String> connection = (StatefulRedisClusterConnection) this.pool
                    .borrowObject();
String value = (String) connection.sync().get(id);
connection.close();
pool.clear();
redisClient.shutdown();

<details>
<summary> stackTrace:
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:577)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:476)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)</summary>

Below is a screenshot of how many such threads are active in our server
![image](https://user-images.githubusercontent.com/41999937/158509173-60462b06-b4e8-4098-85a4-454ac62f7521.png)

Lettuce Version : 6.1.6.RELEASE
LatencyUtils : 2.0.3
HarishMalavade commented 2 years ago

image

mp911de commented 2 years ago

You're creating ClientResources that need to be shut down after usage.

HarishMalavade commented 2 years ago

@mp911de thanks for the response. The threads are still active even if I dont use client resource. doesnt client.shutdown() shutdown the resource as well? Could there be any other place where I need to disable or shutdown few things?