Open tgpfeiffer opened 10 years ago
I examined this issue a bit further, and it seems like the following code is necessary to shut down all Netty threads properly when using netty 3.6.6:
client.close()
client.getEventLoop.shutdown()
client.getEventLoop.asInstanceOf[NettyEventLoop].getClientFactory.shutdown()
Just calling client.close()
will leave us with
Thread[Hashed wheel timer #1,5,main] - Hashed wheel timer #1, 26, TIMED_WAITING, false
Thread[New I/O boss #17,5,main] - New I/O boss #17, 27, RUNNABLE, false
Thread[New I/O worker #1,5,main] - New I/O worker #1, 10, RUNNABLE, false
...
Thread[New I/O worker #16,5,main] - New I/O worker #16, 25, RUNNABLE, false
Thread[pool-3-thread-1,5,main] - pool-3-thread-1, 28, TIMED_WAITING, false
Calling just client.getEventLoop.shutdown()
leads to remaining threads
Thread[Hashed wheel timer #1,5,main] - Hashed wheel timer #1, 26, TIMED_WAITING, false
Thread[New I/O boss #17,5,main] - New I/O boss #17, 27, RUNNABLE, false
Thread[New I/O worker #1,5,main] - New I/O worker #1, 10, RUNNABLE, false
...
Thread[New I/O worker #16,5,main] - New I/O worker #16, 25, RUNNABLE, false
(note that the "pool-3-thread-1" is gone) and just client.getEventLoop.asInstanceOf[NettyEventLoop].getClientFactory.shutdown()
will result in
Thread[pool-1-thread-1,5,main] - pool-1-thread-1, 10, TIMED_WAITING, false
...
Thread[pool-1-thread-16,5,main] - pool-1-thread-16, 25, TIMED_WAITING, false
In all three cases, the program will not terminate when run from within sbt or an IDE. The combination of all three lines is required to exit cleanly.
However, the shutdown()
method was added to Netty only in 3.6 (see netty/netty@7aa2cfad65ca6e9d7cbdd0f07ce3d13de861553d), it seems. Can we somehow incorporate calling this method, thereby also allowing to use newer versions of Netty?
I am working on a Scala project that uses both Apache Spark (http://spark.apache.org/) and msgpack-rpc. Spark (transitively) depends on
io.netty:netty:3.6.6.Final
while msgpack-rpc depends onorg.jboss.netty:netty:3.2.1.Final
.If I just include the dependencies to Spark and msgpack-rpc in my build.sbt file, then Spark will refuse starting with
So I edited my sbt file and excluded the transitive dependency on
org.jboss.netty:netty
from the msgpack-rpc dependency; that is, the (newer) netty version 3.6.6.Final is used. msgpack-rpc will still run fine, but has problems with shutdown:client.getEventLoop.shutdown()
will not stop the Netty threads, a lot of "New I/O worker" and one "New I/O boss" thread will stay alive.This is a problem, for instance, when using
sbt run
, because this will wait for all non-daemon threads to exit before exiting/returning to the sbt shell. This is never the case with the newer Netty version.So I was wondering if it's possible to update the msgpack-rpc code to also work with a newer Netty version when it comes to thread shutdown. Or is it in fact a Netty bug?