ntua-unit-of-control-and-informatics / jaqpot-api-archived

JAQPOT Quattro is the 4th version of a YAQP, a RESTful web service which can be used to train machine learning models and use them to obtain toxicological predictions for given chemical compounds or engineered nano materials. The project is written in Java8 and JEE7.
Other
8 stars 4 forks source link

Zombie threads of cancelled tasks #27

Open alphaville opened 9 years ago

alphaville commented 9 years ago

Thread.interrupt() will not stop a thread: in TaskResource:L173 we issue a Thread.interrupt() to the thread that runs the specified task. However, this will not stop the thread - it will merely trigger Thread.isInterrupted() which needs to be handled from within the running thread. What we need to do, is to go to the various MDBs (e.g., TrainingMDB) and add this piece of code at various positions of the code where we allow it to interrupt:

if (shouldStop()) {
   return;
}

where

private boolean shouldStop() {
   return Thread.currentThread.isInterrupted();
}

Unless we do that, the thread will not be really interrupted. Mind also, that stopping threads in Java doesn't come out of the box. This post explains that stopping a thread is not a trivial task.

hampos commented 9 years ago

The solution is not ideal. If it is not possible to listen for an interruption event from the MDB and return, then we should abandon the idea of task cancelling. Messing with threads that are out of our scope is very dangerous anyway. Check this question: http://stackoverflow.com/questions/533783/why-is-spawning-threads-in-java-ee-container-discouraged

alphaville commented 9 years ago

But we are not creating any threads. We just keep track of the threads that are created by the container to be able to interrupt them.

hampos commented 9 years ago

From the EJB 3.0 specification: The enterprise bean must not attempt to manage threads. The enterprise bean must not attempt to start, stop, suspend, or resume a thread, or to change a thread’s priority or name. The enterprise bean must not attempt to manage thread groups.

alphaville commented 9 years ago

I see... but we then need to find a proper way to interrupt those threads when necessary (when a client issues a DELETE on a task).

hampos commented 9 years ago

If we really need such functionality, I believe we must rethink about using this: http://docs.oracle.com/javaee/7/tutorial/batch-processing.htm#GKJIQ6

Lets use the initial solution for now if it does not harm the JMS threads and discuss it in more detail later.

hampos commented 8 years ago

Been working on this issue, training and prediction tasks are now cancellable due to the use of the new async JPDIClient. One can cancel a task when it waits for responses from algorithm services, but cannot do so when it's making database operations, or requests on the main API. Planning on implementing an async client for the main API that will allow for cancelling as well.