sermant-io / Sermant

CNCF sandbox project, a Cloud-Native Proxyless Service Mesh based on Java Bytecode Enhancement Technology
https://sermant.io/
Apache License 2.0
1.25k stars 164 forks source link

【bug】当Sermant长时间无法和backend-lite建立连接,心跳服务将无法再上报心跳 #949

Closed luanwenfei-venus closed 1 year ago

luanwenfei-venus commented 1 year ago

What happened?

开启心跳后,如果长时间没有连接上后端,将无法上报心跳,并抛出如下的异常 Exception in thread "Thread-1" java.util.concurrent.RejectedExecutionException: Task com.huaweicloud.sermant.implement.service.send.NettyClient$$Lambda$24/1083915630@6d2252b0 rejected from java.util.concurrent.ThreadPoolExecutor@3ed4f214[Running, pool size = 10, active threads = 10, queued tasks = 1000, completed tasks = 1313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at com.huaweicloud.sermant.implement.service.send.NettyClient.sendData(NettyClient.java:174) at com.huaweicloud.sermant.implement.service.heartbeat.HeartbeatServiceImpl.execute(HeartbeatServiceImpl.java:127) at com.huaweicloud.sermant.implement.service.heartbeat.HeartbeatServiceImpl.access$000(HeartbeatServiceImpl.java:53) at com.huaweicloud.sermant.implement.service.heartbeat.HeartbeatServiceImpl$1.run(HeartbeatServiceImpl.java:87) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750)

How can we reproduce it (as minimally and precisely as possible)?

可以将启动sermant并且开启心跳服务,将心跳上报时间改为1000ms,断开十分钟左右,将重现此问题

Anything else we need to know?

No response

Sermant version

0.7.0

OS version

```console # On Linux: $ cat /etc/os-release # paste output here $ uname -a # paste output here # On Windows: C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture # paste output here ```
github-actions[bot] commented 1 year ago

This issue seems to be Stale. We will close it in a few days.

github-actions[bot] commented 1 year ago

We close this issue because it hasn't been updated in a while. Remove Stale label if you want to reopen it.