Hello, here are the errors reported by my program:
My service is deployed in k8s, with 5 pods. After running for 1 day, 1 to 2 pods will have the following errors. Other pods are running normally, but over time, they will also have errors
2023-03-29 10:22:41.716 [ERROR] [pid] [] [] [] [redisson-timer-6-1] [o.r.c.handler.PingConnectionHandler 97] Unable to send PING command over channel: [id: 0xa34aa017, L:/10.90.140.103:53134 - R:10.1.128.56/10.1.128.56:7379]
org.redisson.client.RedisTimeoutException: Command execution timeout for command: (PING), params: [], Redis client: [addr=redis://10.1.128.56:7379]
at org.redisson.client.RedisConnection.lambda$async$0(RedisConnection.java:244)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:668)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:743)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:471)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2023-03-29 10:44:15.353 [ERROR] [pid] [] [] [] [redisson-netty-4-37] [o.r.c.SentinelConnectionManager 372] Can't execute SENTINEL commands on 10.1.128.55/10.1.128.55:27379
java.util.concurrent.CompletionException: org.redisson.client.RedisTimeoutException: Command execution timeout for command: (SENTINEL GET-MASTER-ADDR-BY-NAME), params: [sentinel_cs], Redis client: [addr=redis://10.1.128.55:27379]
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:975)
at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124)
at org.redisson.misc.CompletableFutureWrapper.thenCompose(CompletableFutureWrapper.java:193)
at org.redisson.connection.SentinelConnectionManager.checkMasterChange(SentinelConnectionManager.java:500)
at org.redisson.connection.SentinelConnectionManager.updateState(SentinelConnectionManager.java:358)
at org.redisson.connection.SentinelConnectionManager.lambda$checkState$4(SentinelConnectionManager.java:351)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:778)
at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2140)
at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:110)
at org.redisson.connection.SentinelConnectionManager.checkState(SentinelConnectionManager.java:343)
at org.redisson.connection.SentinelConnectionManager.access$400(SentinelConnectionManager.java:48)
at org.redisson.connection.SentinelConnectionManager$2.run(SentinelConnectionManager.java:321)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:125)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:465)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
my pom.xml
spring-boot-starter-parent 2.0.3.RELEASE
redisson 3.17.6
spring-data-redis 2.0.8
public RedissonClient redissonSentinel() {
Config config = new Config();
String[] nodes = redissonSentinelConfig.getNodes().split(",");
List<String> nodeList = new ArrayList(nodes.length);
Arrays.stream(nodes).forEach(
(index) -> nodeList.add(index.startsWith("redis://") ? index : "redis://" + index));
SentinelServersConfig serverConfig = config.useSentinelServers()
.addSentinelAddress(nodeList.toArray(new String[0]))
.setMasterName(redissonSentinelConfig.getMaster())
.setDatabase(redissonSentinelConfig.getDatabase())
.setTimeout(6000)
.setPingConnectionInterval(10000)
.setSubscriptionsPerConnection(5000)
.setSubscriptionConnectionPoolSize(1000);
if (StringUtils.isNotBlank(redissonSentinelConfig.getPassword())) {
serverConfig.setPassword(redissonSentinelConfig.getPassword());
}
config.setNettyThreads(64);
return Redisson.create(config);
}
Hello, here are the errors reported by my program: My service is deployed in k8s, with 5 pods. After running for 1 day, 1 to 2 pods will have the following errors. Other pods are running normally, but over time, they will also have errors
my pom.xml spring-boot-starter-parent 2.0.3.RELEASE redisson 3.17.6 spring-data-redis 2.0.8