spring-cloud / spring-cloud-netflix

Integration with Netflix OSS components
http://cloud.spring.io/spring-cloud-netflix/
Apache License 2.0
4.86k stars 2.44k forks source link

Threads not shutdown on tomcat shutdown #1688

Closed Ziemowit closed 7 years ago

Ziemowit commented 7 years ago

Hi we are using: spring-boot: 1.4.4 spring-cloud: CAMDEN.SR5 tomcat: 7.0.59

We use embedded Tomcat and when kill application via actuator endpoint at the end of killing we receive in logs:

2017-02-13 11:53:50,584 | Thread-9   |              | INFO  | alContainerEntityManagerFactoryBean | Closing JPA EntityManagerFactory for persistence unit 'default'
2017-02-13 11:53:50,589 | Thread-9   |              | INFO  | Http11NioProtocol                   | Pausing ProtocolHandler ["http-nio-8080"]
2017-02-13 11:53:50,640 | Thread-9   |              | INFO  | AjpProtocol                         | Pausing ProtocolHandler ["ajp-bio-8009"]
2017-02-13 11:53:50,641 | Thread-9   |              | INFO  | StandardService                     | Stopping service Tomcat
2017-02-13 11:53:50,665 | tartStop-2 |              | INFO  | [...]                       | Destroying Spring FrameworkServlet 'dispatcherServlet'
2017-02-13 11:53:50,668 | tartStop-2 |              | INFO  | SessionReporter                     | http session destroyed, total = 64
...
2017-02-13 11:53:50,693 | tartStop-2 |              | ERROR | WebappClassLoader                   | The web application [...] appears to have started a thread named [RxScheduledExecutorPool-1] but has failed to stop it. This is very likely to create a memory leak.
2017-02-13 11:53:50,693 | tartStop-2 |              | ERROR | WebappClassLoader                   | The web application [...] appears to have started a thread named [RxScheduledExecutorPool-2] but has failed to stop it. This is very likely to create a memory leak.
2017-02-13 11:53:50,694 | tartStop-2 |              | ERROR | WebappClassLoader                   | The web application [...] appears to have started a thread named [RxIoScheduler-1 (Evictor)] but has failed to stop it. This is very likely to create a memory leak.
2017-02-13 11:53:50,694 | tartStop-2 |              | ERROR | WebappClassLoader                   | The web application [...] appears to have started a thread named [RxComputationScheduler-1] but has failed to stop it. This is very likely to create a memory leak.
2017-02-13 11:53:50,695 | tartStop-2 |              | ERROR | WebappClassLoader     
2017-02-13 11:53:50,698 | tartStop-2 |              | ERROR | WebappClassLoader                   | The web application [...] appears to have started a thread named [hystrix-...-common-3] but has failed to stop it. This is very likely to create a memory leak.

The problem is in memory leaks. Now maybe you are able to help me... Should I add some custom behaviour for the application when it is closing? All components which cannot be "killed" seems to have some point of contact in @FeignClient.

spencergibb commented 7 years ago

There are when you call shutdown?

Ziemowit commented 7 years ago

Yes. When I call /shutdown endpoint then application closes itself but by the way at the end I get the list of errors "failed to stop".

It happens only for one of whole bunch of my services. The others (simpler one) I am able to kill without any errors. In problematic service I have much more extra components/libraries/etc. I had hope that maybe there is some known issue that spring boot interferes with some component but seems that not :)

Then I need to hmmm switching off components one by one and checking if /shutdown still generates errors.

I will find out it tomorrow and make my question more specific!

Thank you for your interest!

spencergibb commented 7 years ago

It's not fatal at that point.

spencergibb commented 7 years ago

Closing, duplicate of #106

Merry1227 commented 7 years ago

@spencergibb , @dsyer I am seeing the same warnings in our environment. When I look up to the implementation of DataPublisher in com.netflix.stats.distribution and the usage class ServerStats. I really found the close() method which will call DataPublisher.stop to stop the long-running periodical thread should be called when spring shutdown. But It seems actually not!!! Cloud you confirm it? Thank you in advance. 2017-08-25 10:51:29.044 WARN 5 --- [localhost-startStop-2] o.a.c.loader.WebappClassLoaderBase : The web application [ROOT] appears to have started a thread named [DataPublisher] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:

anks14 commented 5 years ago

This issue seems to be happening for spring boot : 2.1.3.RELEASE spring cloud: Greenwich.SR1

when using Hystrix and calling actuator/shutdown url. Any one find a resolution to this issue?

ryanjbaxter commented 5 years ago

Open an issue with a sample that reproduces the problem