ProjectZetta / RemoteFutures

Remote execution of futures in a distributed system.
http://remotefutures.org
Other
4 stars 1 forks source link

[Docker] java.util.concurrent.RejectedExecutionException: Worker has already been shutdown #62

Closed ghost closed 10 years ago

ghost commented 10 years ago

docker --version Client version: 1.0.0 Client API version: 1.12 Docker image: meduzz/ubuntu_oracle_java8
Java: JDK8u20

File: MultiNodeSampleSpecMultiJvmNode2:

Error: [JVM-2] Sep 09, 2014 4:37:50 PM org.jboss.netty.channel.DefaultChannelPipeline [JVM-2] WARNING: An exception was thrown by an exception handler. [JVM-2] java.util.concurrent.RejectedExecutionException: Worker has already been shutdown

Stacktrace:

[info] * org.remotefutures.pingpong.MultiNodeSampleSpec [JVM-1] Run starting. Expected test count is: 2 [JVM-1] MultiNodeSampleSpecMultiJvmNode1: [JVM-1] A MultiNodeSample [JVM-2] Run starting. Expected test count is: 2 [JVM-2] MultiNodeSampleSpecMultiJvmNode2: [JVM-2] A MultiNodeSample [JVM-1] - must wait for all nodes to enter a barrier [JVM-2] - must wait for all nodes to enter a barrier [JVM-1] Sending ping to ponger [JVM-2] - must send to and receive from a remote node [JVM-1] - must send to and receive from a remote node [JVM-2] Sep 09, 2014 4:37:50 PM org.jboss.netty.channel.DefaultChannelPipeline [JVM-2] WARNING: An exception was thrown by an exception handler. [JVM-2] java.util.concurrent.RejectedExecutionException: Worker has already been shutdown [JVM-2] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120) [JVM-2] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72) [JVM-2] at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) [JVM-2] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56) [JVM-2] at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) [JVM-2] at org.jboss.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) [JVM-2] at org.jboss.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) [JVM-2] at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) [JVM-2] at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) [JVM-2] at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) [JVM-2] at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) [JVM-2] at org.jboss.netty.channel.Channels.close(Channels.java:812) [JVM-2] at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:197) [JVM-2] at akka.remote.testconductor.ClientFSM$$anonfun$5.applyOrElse(Player.scala:266) [JVM-2] at akka.remote.testconductor.ClientFSM$$anonfun$5.applyOrElse(Player.scala:264) [JVM-2] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36) [JVM-2] at akka.actor.FSM$class.terminate(FSM.scala:672) [JVM-2] at akka.actor.FSM$class.postStop(FSM.scala:658) [JVM-2] at akka.remote.testconductor.ClientFSM.postStop(Player.scala:145) [JVM-2] at akka.actor.Actor$class.aroundPostStop(Actor.scala:475) [JVM-2] at akka.remote.testconductor.ClientFSM.aroundPostStop(Player.scala:145) [JVM-2] at akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210) [JVM-2] at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172) [JVM-2] at akka.actor.ActorCell.terminate(ActorCell.scala:369) [JVM-2] at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462) [JVM-2] at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478) [JVM-2] at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263) [JVM-2] at akka.dispatch.Mailbox.run(Mailbox.scala:219) [JVM-2] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) [JVM-2] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [JVM-2] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [JVM-2] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [JVM-2] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:

MartinSenne commented 10 years ago

Hmmm, sbt

multi-jvm:test-only org.remotefutures.pingpong.MultiNodeSampleSpec works.

ghost commented 10 years ago

I am still unsure if this is due to some unlucky timing..

Means, it could be that the docker container may not get enough CPU horsepower on my oldy laptop to ensure proper timing between multi-JVM's :-(

Anyway, I am rebuilding the container at the moment for investigating this issue further.

Hmmm, sbt

multi-jvm:test-only org.remotefutures.pingpong.MultiNodeSampleSpec works.

— Reply to this email directly or view it on GitHub https://github.com/DistributedRemoteFutures/DistributedRemoteFutures/issues/62#issuecomment-55000090 .

ghost commented 10 years ago

In Docker, sbt multi-jvm:test-only org.remotefutures.pingpong.MultiNodeSampleSpec

reproduces the error BUT all tests are reported as passed.

multi-jvm:test-only org.remotefutures.pingpong.MultiNodeSampleSpec [info] * org.remotefutures.pingpong.MultiNodeSampleSpec [JVM-1] Run starting. Expected test count is: 2 ... [JVM-2] Sep 09, 2014 6:36:11 PM org.jboss.netty.channel.DefaultChannelPipeline [JVM-2] WARNING: An exception was thrown by an exception handler. [JVM-2] java.util.concurrent.RejectedExecutionException: Worker has already been shutdown .....

ushAndStop
[JVM-2] Run completed in 5 seconds, 287 milliseconds.
[JVM-2] Total number of tests run: 2
[JVM-2] Suites: completed 1, aborted 0
[JVM-2] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[JVM-2] All tests passed.
[JVM-1] Run completed in 5 seconds, 316 milliseconds.
[JVM-1] Total number of tests run: 2
[JVM-1] Suites: completed 1, aborted 0
[JVM-1] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[JVM-1] All tests passed.
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 9 s, completed Sep 9, 2014 6:36:12 PM