search

OpenLiberty / open-liberty

Open Liberty is a highly composable, fast to start, dynamic application server runtime environment
https://openliberty.io
Eclipse Public License 2.0
1.15k stars 592 forks source link

Liberty server hangs infinitely when stopping #8741

Open milanshergill opened 5 years ago

milanshergill commented 5 years ago

We have deployed Spring application on Liberty server. We stop the Liberty server using server stop <servername>. Now the issue we are facing is that one of the deployed Spring application has a Retryable method in it, which in some cases keeps trying. In this scenario when we shutdown the server, it hangs infinitely. Any help would be appreciated, thanks!

This issue was discussed on #was-liberty slack channel and Tom Watson did initial investigation to help me with this.

I created the server dump and provided it to Tom for his review. I have also uploaded the ThreadInfoIntrospector.txt file with this issue.

Tom's initial investigation reports this: It looks like there is a deadlock between com.ibm.ws.webcontainer.webapp.WebApp.removeExtensionFactory and com.ibm.ws.webcontainer.webapp.WebApp.initialize [ThreadInfoIntrospector.txt](https://github.com/OpenLiberty/open-liberty/files/3535794/ThreadInfoIntrospector.txt)

tjwatson commented 5 years ago

We may be tempted to blame Spring Boot here. And I would agree that Spring Boot is blocking the application from completing initialization with some indefinite number of sleeps from the app installation thread . But I would hope such an application would not cause deadlock when trying to force the server to stop.

pmd1nh commented 5 years ago

hi, Can I request a zipped up server with the app to try this?

Since the app just goes to sleep during the startup, it has never finished the initialization. I think I can check on that init status and ignore this app during the server shutdown. I just need a way to recreate this though.

pnicolucci commented 5 years ago

@tjwatson @milanshergill can either of you help with @pmd1nh request?