Timeout after [300] seconds waiting for service container stability

[dklesev]

If I run wildfly in docker and redeploy a WAR (several times) I get following:

15:45:39,879 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 12) WFLYCTL0348: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'full-replace-deployment' at address '[]'
15:45:44,883 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 12) WFLYCTL0190: Step handler org.jboss.as.server.deployment.DeploymentHandlerUtil$4@199bade6 for operation {"operation" => "full-replace-deployment","address" => [],"name" => "myApplication.war","content" => [{"input-stream-index" => 0}],"enabled" => true,"operation-headers" => {"caller-type" => "user","access-mechanism" => "NATIVE"},"runtime-name" => undefined,"persistent" => true,"owner" => undefined} at address [] failed handling operation rollback -- java.util.concurrent.TimeoutException: java.util.concurrent.TimeoutException
    at org.jboss.as.controller.OperationContextImpl.waitForRemovals(OperationContextImpl.java:498)
    at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1369)
    at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
    at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1301)
    at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
    at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
    at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:644)
    at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
    at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1329)
    at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:400)
    at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:222)
    at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler.doExecute(ModelControllerClientOperationHandler.java:208)
    at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler.access$300(ModelControllerClientOperationHandler.java:130)
    at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run(ModelControllerClientOperationHandler.java:152)
    at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run(ModelControllerClientOperationHandler.java:148)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:149)
    at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1.execute(ModelControllerClientOperationHandler.java:148)
    at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
    at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)

15:45:49,884 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 12) WFLYCTL0349: Timeout after [5] seconds waiting for service container stability while finalizing an operation. Process must be restarted. Step that first updated the service container was 'full-replace-deployment' at address '[]'

The data dir is shared with host (docker swarm mode). I'm really out of ideas what can be the case btw. how to solve this...

[hhimanshu]

I am facing a similar issue

[balarambarada]

i also faced the similar issue for Azure - window 16 VM and solved after adding the below line .

below line to be added in standalone.conf.bat

set "JAVA_OPTS=-Djboss.as.management.blocking.timeout=600"

[dhanraj-popalghat-sp]

I'm also facing this issue. Is there any solution? Did above solution work?

[balarambarada]

Fot us ,we added RAM capacity on that VM . then its working fine

[Mahn00rMalik]

I am also facing a similar issue.

2019-11-29 16:15:12,084 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0348: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[ ("core-service" => "management"), ("management-interface" => "http-interface") ]'

I have added RAM as well. I also tried the following set "JAVA_OPTS=-Djboss.as.management.blocking.timeout=600" still facing the same issue

[NorthPoleLars]

I'm also facing this issue. Is there any solution? Did above solution work?

Find any solution?

[alexey2baranov]

i also faced the similar issue for Azure - window 16 VM and solved after adding the below line .

below line to be added in standalone.conf.bat

set "JAVA_OPTS=-Djboss.as.management.blocking.timeout=600"

where is this file located? Can you explain how to set this option inside docker-compose.yml? I will very appreciate it. thanks

[wrmuchefa]

where is this file located? Can you explain how to set this option inside docker-compose.yml? I will very appreciate it. thanks

/{KEYCLOAK_HOME}/bin/standalone.conf

[rafayk98]

Same question I need to change this setting in a docker-compose file how do I do it directly from my docker-compose file?

[jcartw]

Had a similar issue related to mysql in docker running slow on a ext4 drive that I mounted. Fixed it by updating the options in /etc/fstab (i.e. barrier=0). See https://phpforus.com/how-to-make-mysql-run-fast-with-ext4-on-ubuntu/.