Open a-filimonov opened 1 year ago
the same got with eureka-client version 1.10.17 Spring Cloud version: 3.1.7
same error in our production environment with eureka-client version 2.0.1 Spring Cloud version: 2023.0.0 Spring Boot: 3.2.4
logger_name com.netflix.discovery.TimedSupervisorTask
java.util.concurrent.TimeoutException: null
at java.base/java.util.concurrent.FutureTask.get(Unknown Source)
at com.netflix.discovery.TimedSupervisorTask.run(TimedSupervisorTask.java:68)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
We have an issue with one of the pods in cluster became unreachable due to the fact that Eureka client did not update the service registry cache. Initially this pod started to log WARNs that the thread pool is full:
Then after 4 days, the dependent service was redeployed, which caused hosts to change, however the affected pod failed to get the updated list of hosts, and started to fail all the requests, since the dependent service is used in every request.
This caused a temporary outage on production system until we noticed the pod under issue and rebooted it. All other pods of the same system did not have this issue and were stable.
After some investigation, it seems that this issue was reproducible couple of times on different versions of eureka-client: For reference: https://github.com/Netflix/eureka/issues/907
The version of eureka-client is at the moment: 1.10.17 Spring Cloud version: 3.1.2
Also we noticed that eureka-client was updated only in major release of Spring Cloud: https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-netflix-eureka-client/4.0.0