search

eclipselabs / passerelle

Passerelle, an actor-based process engine based on Ptolemy II
5 stars 3 forks source link

Review throttling behaviour between actors in thread-based domain #63

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 

DAWN has reported that the expected "streaming" mode behaviour to send large 
sets of large files through a flow no longer works since their lasst update. 
I.e. the model risks going out-of-memory when the actual file processing takes 
too long.

The desired behaviour would be to be able to configure receiver queues with 
limited size and an actors "send" operation block until the receiver queue has 
room.

We need to review the detailed behaviour of receivers and queues in input 
ports. Receiver queues should support setting size limitations (should already 
be OK) and different types of rejection handlers.

Original issue reported on code.google.com by erwin...@gmail.com on 9 Oct 2013 at 11:25

GoogleCodeExporter commented 9 years ago 
Mark Basham has also hit this bug with the current DAWN workflows version. It 
definitely used to work because we used to process large directories of files, 
now we cannot. It may be that the DataImportSource has been broken or does not 
work correctly in the new framework. I.e. not a passerelle problem but an 
assumption made by an actor working in an old passerelle version.

Original comment by matthew....@gmail.com on 11 Oct 2013 at 9:10

GoogleCodeExporter commented 9 years ago 
I've created two trivial unit tests for a limited receiver queue size : one 
that refuses extra msgs and one that lets everything through but logs a warning.
They both work as expected.

But : refusing extra messages is not what you need. Your use case needs to 
block the sender actor until there is room available at the receiver side.

Also, these queue limits only work for blocking input ports (PULL mode), not 
for PUSH-mode ports.

So I'll add extra behaviours to match your use case.

Remark that SVN commits are currently not working, so the new test code is not 
yet on the repos...

Original comment by erwin...@gmail.com on 11 Oct 2013 at 3:23

GoogleCodeExporter commented 9 years ago

Original comment by erwin...@gmail.com on 5 May 2014 at 2:54