concurrency issue in queue execution

[GoogleCodeExporter]

 public void completeExecution (NodeToken token, String arcName)
  {
    GraphProcess process = token.getProcess();

    if ( !process.isExecuting() )
    {
      return;
    }

    completeNodeToken( token, arcName, false );

    if ( !arcExecutionStarted )
    {
      executeQueuedArcTokens( process );
    }
  }

  @Override
  public void executeQueuedArcTokens (GraphProcess process)
  {
    arcExecutionStarted = true;

    try
    {
      while ( !process.isArcTokenQueueEmpty() )
      {
        executeArc( process, process.dequeueArcTokenForExecution() );
      }
----------> here
      checkForCompletion( process );
    }
    finally
    {
      arcExecutionStarted = false;
      drainAsyncQueue( process );
    }
  }

--------

It happens that some thread calls completeExecution while another thread
already exited the while loop. Since arcExecutionStarted remains true, the
other thread doesn't enter executeQueuedArcTokens and elements remain in
the queue.

This freezes the process execution in my case.

Am I using it incorrectly? Shall this code be fixed to guarantee that there
is always some worker that executes arcs? Which thread is preferred? The
one that was already consuming tokens or the new one which called
completeExecution?

I'm willing to fix the code and post you a patch, after having tested it
etc, but I'd like to hear your opinion first.

I'm using 1.0.0-rc2 but I also took a look to the latest svn code and the
situation doesn't seem changed.

Original issue reported on code.google.com by mmikuli...@gmail.com on 19 Nov 2009 at 6:13

[GoogleCodeExporter]

Engine instances are not designed to be thread safe. This is especially true of 
the
HibEngine, which wraps a non-thread-safe hibernate session. But all subclasses 
of
BaseEngine use local state, which as you note, does not cope well with 
multi-threaded
use. 

Engine instances should be sufficiently lightweight that you can create one for 
each
thread that needs access.

I will update the javadoc to make this more clear. Let me know if this helps.

Original comment by plor...@gmail.com on 19 Nov 2009 at 6:41

[GoogleCodeExporter]

does this mean that I can use more than one engine to process arc tokens of the 
same
process?

I need a way to "wake up" a process waiting on asynchronous events. I didn't 
want to
dedicate a single thread to processes, but wanted to "steal" time from other 
threads,
as needed.

Original comment by mmikuli...@gmail.com on 22 Nov 2009 at 11:28

[GoogleCodeExporter]

Yes, I usually just create an engine instance whenever I need one, in whatever 
thread
I need one in. The only caveat is that you should only work with a given 
process with
one thread at a time. So if you know this is the case, then great. Otherwise you
should lock the process. In the memory engine, you could synchronize on it. In 
the
hibernate engine, you could call session.lock on it.

Original comment by plor...@gmail.com on 23 Nov 2009 at 3:25

[GoogleCodeExporter]

unfortunately this approach doesn't work for me. I would like to enqueue the 
token
and let the original thread continue the processing. Otherwise I would have two
threads blocked on the execution of the process, while only one is actually 
necessary.

I can work for a solution with a thread safe queue handling for the memory 
engine
(perhaps a subclass?) and then send you the patches.

Original comment by mmikuli...@gmail.com on 23 Nov 2009 at 2:13

[GoogleCodeExporter]

To clarify, you need to be able to execute the same process in two separate 
threads
at the same time?

Original comment by plor...@gmail.com on 23 Nov 2009 at 2:19

[GoogleCodeExporter]

not really, I just need to be able to enqueue a "completion" of a token from 
thread
A, while thread B is possibly executing code on behalf of the same process.

Original comment by mmikuli...@gmail.com on 23 Nov 2009 at 2:29

[GoogleCodeExporter]

.. and I don't really care which of the two threads (A or B) will continue 
processing
the tokens (which is triggered if the queue is empty).

Currently if the queue is empty, and thread B "completes" a token, then the 
queue
will be drained in the context of thread B. For me this is fine.

If there are still some elements in the queue, but thread A is processing the 
same
queue, then adding a token to the queue should work (assuming that a thread safe
queue implementation is used behind the scenes). Thread B will exit 
immediately, and
thread A will have just another thing to process.

The problem is that thread A could have drained the queue but still hold the
arcExecutionStarted boolean to true. If thread B adds something to the queue at 
that
point (after the 'while' loop which drains the queue), it's too late for thread 
A to
continue draining the queue, and since "accExecutionStarted" is still true, 
thread B
will exit leaving one element in the queue that nobody will process, until 
something
happens.

Original comment by mmikuli...@gmail.com on 23 Nov 2009 at 2:36

[GoogleCodeExporter]

Ok, I think I understand what you're after. I don't think that will be easy to
implement, as you're going to have to be careful locking the arc token queue. 
But I'm
happy to look at patches. 

I tend not to worry about locked processes. The nodes only do a small amount of 
set
up work, the rest is done asynchronously, so process execution is fast.

Original comment by plor...@gmail.com on 23 Nov 2009 at 3:10

[GoogleCodeExporter]

Original comment by plor...@gmail.com on 28 Apr 2010 at 1:23

• Changed state: WaitingForPatch