Default strategy of assignment of work for workers should be a shared queue.

[sbragagnolo]

Now there is an master worker that takes care about the pool, and assigns tasks to the different workers, each of them has a different queue. This leads to many problematics:

• Once the scheduled tasks are divided, there cannot be easily re-assigned,
• If a worker dies in the process, to spawn a new o equvialent process is an important matter (since the worker may have in it state many future tasks
• Is hard to keep track the task that are not left executed.
• In order to arrive to a good order of excecution we should analyse the tasks before assign them (now adays we only split them in equal amounts)

=======

A common queue would resolve all this problems, leading to an organic load balance.

[guillep]

Hmm, but if a worker dies while it was working on a task, it has the same problem.

This actually means that we have to keep track of:

• what are the tasks deployed in a worker
• check frequently what is the status of each worker (and so if it is dead we can re-schedule it)

In any case, this is not a problem of the worker, it is the problem of the worker pool :)

[sbragagnolo]

Yeah, I should tag this as worker pool instead of worker. For me are both in the same bag :P.

Yeah, it's mainly a deal of the worker pool, we just need to set up the same queue to each worker and its done that part.

Then change the pool worker from master to watchdog. The watchdog can staring an 'executing' queue, with task executions, so it can know the state, the task etc.

• If the worker dies because a TKT error, then the task should be re-scheduled
• If the worker dies by other causes (like killed by closing the debugger), the task execution should be removed from executing, the worker removed from the known workers of the pool, so a new worker will be spawned when needed.

[sbragagnolo]

Added. Now we have two flavours of pool