Option to make a job recurreing

[sagivf]

Great project @grantcarthew, I'd really like to start using it.

Is there any way to make a job recurring or better yet - recurring with an option to function that will check whether to stop the recurrence.

[grantcarthew]

@sagivf I don't think that fits into the job description of a queue.

In your project code you could simply process the job using rethinkdb-job-queue, then when the job is completed, check to see if it needs to run again, then add it.

Don't forget that you can delay the processing of jobs also. So if you do need the job processed again, on completion, add it with a delay (if needed).

Delayed Job: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Delayed-Job

[sagivf]

That's a good solution, I'll do that.

I still think It would be nice to have it out of box. Perhaps it could be a separate type. Something like an IntervalQueue, needs more thought...

It's especially nice to have cron type jobs run out of the box. Kind of like what agenda supports - https://github.com/rschmukler/agenda#creating-jobs

[grantcarthew]

OK, I had a read though the Agenda doc. I'll consider it for v2. I have added it to the v2 Project: https://github.com/grantcarthew/node-rethinkdb-job-queue/projects/1

I am a strong supporter of the KISS principle and this may be moving away from that.

[sagivf]

I agree that KISS is the best. But seeing as this is a very common use case I think it would be handy. It might also help in optimizing all sort of concurrency and parallelism issues that may arise. I'll try and help out if I can, I'm a little overloaded at the moment 😵

[grantcarthew]

@sagivf I've been thinking on this and I don't think it would be too hard to add the functionality to repeat jobs. Because it may be simple to add I am considering it more so.

My thoughts are:

• Add a new property to jobs such as repeatJob which supports three values:
  • false === Normal job processing.
  • true === Repeats immediately upon job completion.
  • Number === Delay in milliseconds after job completion before restart.
• On job completion or job failure (triggered by the next() call), check for the repeatJob value and log/set properties as needed.

I'm not sold on it yet though. I still think the role of rethinkdb-job-queue is to queue jobs and process them. It is a building block, not a framework.

[grantcarthew]

Maybe two properties: repeatJob and repeatDelay. Then you could set repeatJob to a number of repeats.

[TomKaltz]

I'm thinking @sagivf was referring to adding more of a full features cron-type scheduler just like https://github.com/rschmukler/agenda.

I do however like your proposal regarding repeatJob and repeatDelay @grantcarthew. Sounds like it will add usable functionality without too much complexity.

[grantcarthew]

I've already started work on this. Adding Job repeat and repeatDelay properties.

[Globik]

But agenda within has a polling mechanism for a new data from db. Is rethinkdb-job-queue also based on the polling requests to rethinkdb?

[grantcarthew]

Hi @Globik. Yes there is a polling process. It can't be avoided. At the moment, polling is used to detect jobs that have not completed due to a hung process. It also removes completed jobs and initiates processing on delayed jobs. Once the repeat feature is complete it will initiate delayed repeat jobs also.

Read more here: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue-Master

[Globik]

Нi @grantcarthew , thank you for explaining, i'm just newbie in these questions. Many dreaming about und saying, that polling can be avoided with server sent notifications. But I also in doubt, am thinking also that it is hardly to be avoided of jobs polling.

[grantcarthew]

This feature is complete and will be published with v2.0. I'll leave the issue open until v2.0 is published.

[grantcarthew]

Not recurreing but repeatable in v2.0. Now published on NPM.

Two new Job options not yet documented:

• Job.repeat either true to repeat for ever, or a number.
• Job.repeatDelay ms before the job repeats.

Note that if you set the repeat value to 2 the job will be processed three times.