Feature: add another option to finish processing

[grantcarthew]

I was helping @ducdigital with issue #48 and realized there is a need to be able to part process a job.

Currently when a job is processed by the Queue.process handler there are only three ways of finishing job processing:

1. Completed
2. Failed
3. Failed with Cancel

This feature would add a new way of finishing job processing for when a job has been part processed and needs to be added back to the queue waiting for next process. At this point in time you can't do this because of the job timeout function.

By adding this feature, complex jobs with multiple steps could be processed by distributed worker nodes.

[grantcarthew]

My first thoughts are to allow passing a partly completed job back into the next() callback.

Within the process handler function you could update properties of the job then call next(null, Job). This would tell the Queue object that you have finished working on the job for now and would like to save it back to the queue.

[TomKaltz]

+1

[ducdigital]

This feature is really good for many use.

Actually I am thinking we can also implement an "ignore" command, in which worker see the queueJob. but does not want to process it and give it back to the queue and let other worker do it.

What do you think?

[grantcarthew]

@ducdigital This is already achieved by the dateEnable job property. If your idea was to only process part A of a job on worker 1 and then part B of a job on worker two, then you would create two queues. When part A of a job is complete, worker one would add the job into the second queue.

[grantcarthew]

FYI This is finished in master. Just need to add tests.

[grantcarthew]

Done. In v2.2.0. I'll leave this open until I finish the doco.

[grantcarthew]

Change log and Queue.process documents updated.