vsivsi / meteor-job-collection

A persistent and reactive job queue for Meteor, supporting distributed workers that can run anywhere.
https://atmospherejs.com/vsivsi/job-collection
Other
385 stars 68 forks source link

Daily running job stops after several executions (runs no longer) #208

Closed a4xrbj1 closed 7 years ago

a4xrbj1 commented 7 years ago

Hi Vaughn,

I've setup two recurring jobs, one is running every hour (to clean completed jobs) and the other one runs every 23h (almost daily I know).

Now after a couple of runs without a problem they both stop (I assume for the hourly one as there are no other jobs running right, so nothing to delete).

But the 23h (daily) job doesn't run for sure, though the last execution is almost 2 days ago. Here's the job:

{ "_id" : "CGM4mPqzWPQLPKdZa", "runId" : null, "type" : "dailyRun", "data" : { "userId" : "qD4hvtrg4kNmApgdj" }, "created" : ISODate("2017-01-07T00:00:06.090Z"), "priority" : 0, "retries" : 1, "retryWait" : 300000, "retried" : 0, "retryBackoff" : "constant", "retryUntil" : ISODate("275760-09-13T00:00:00Z"), "repeats" : 9007199254740986, "repeatWait" : { "schedules" : [ { "h" : [ 0, 23 ] } ], "exceptions" : [ ] }, "repeated" : 6, "repeatUntil" : ISODate("275760-09-13T00:00:00Z"), "depends" : [ ], "resolved" : [ ], "status" : "waiting", "updated" : ISODate("2017-01-07T00:00:06.090Z"), "progress" : { "completed" : 0, "total" : 1, "percent" : 0 }, "log" : [ { "time" : ISODate("2017-01-07T00:00:06.090Z"), "runId" : null, "message" : "Rerunning job", "level" : "info" } ], "after" : ISODate("2017-01-07T23:00:00.005Z") }

This is the code that created this job:

job = new Job(myJobs, 'dailyRun', { userId: userId })
        .repeat({ schedule: myJobs.later.parse.text("every 23 hours") })
        .save();

Can you tell me what's wrong and why they stopped? Can it be because no other job is running at the server (but that was also the case before when they were running last).

As you can see it repeated 6 times and the repeatUntil is set far enough into the future. The time indicated in "after" has passed without the job running again.

Just for completeness, this is the hourly job which ran a lot more times (71x) but seemed to have stopped running 19h later than the daily job:

{ "_id" : "7x9BhbkvsN4AEDgv4", "runId" : null, "type" : "cleanup", "data" : { }, "created" : ISODate("2017-01-07T19:00:13.945Z"), "priority" : 0, "retries" : 1, "retryWait" : 300000, "retried" : 0, "retryBackoff" : "constant", "retryUntil" : ISODate("275760-09-13T00:00:00Z"), "repeats" : 9007199254740921, "repeatWait" : { "schedules" : [ { "h" : [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23 ] } ], "exceptions" : [ ] }, "repeated" : 71, "repeatUntil" : ISODate("275760-09-13T00:00:00Z"), "depends" : [ ], "resolved" : [ ], "status" : "waiting", "updated" : ISODate("2017-01-07T19:00:13.945Z"), "progress" : { "completed" : 0, "total" : 1, "percent" : 0 }, "log" : [ { "time" : ISODate("2017-01-07T19:00:13.945Z"), "runId" : null, "message" : "Rerunning job", "level" : "info" } ], "after" : ISODate("2017-01-07T20:00:00.004Z") }

Here's the code for the hourly job:

job = new Job(myJobs, 'cleanup', {})
        .repeat({ schedule: myJobs.later.parse.text("every 1 hour") }) // change parse.text to the desired interval
        .save();

Thanks in advance for your help.

vsivsi commented 7 years ago

@a4xrbj1 Did you solve this? Jobs only run when there is a worker actively requesting the work. But both of your jobs above say status: 'waiting' which means that the job-collection server doesn't think that all of the conditions (usually elapsed time) have been met for the job status to become 'ready'. I suppose it is possible that the server-side event that "promotes" jobs from waiting to ready when their time has come has stopped running. The easiest way to troubleshoot issues like this is to turn on logging: https://github.com/vsivsi/meteor-job-collection#logging

a4xrbj1 commented 7 years ago

Yes, it was an overlook on my side, mixing up time zones. Anyway, it's running as it should and will explore the logging option.

Thanks as always!

On Tue, 10 Jan 2017 at 03:02, Vaughn Iverson notifications@github.com wrote:

@a4xrbj1 https://github.com/a4xrbj1 Did you solve this? Jobs only run when there is a worker actively requesting the work. But both of your jobs above say status: 'waiting' which means that the job-collection server doesn't think that all of the conditions (usually elapsed time) have been met for the job status to become 'ready'. I suppose it is possible that the server-side event that "promotes" jobs from waiting to ready when their time has come has stopped running. The easiest way to troubleshoot issues like this is to turn on logging: https://github.com/vsivsi/meteor-job-collection#logging

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vsivsi/meteor-job-collection/issues/208#issuecomment-271375275, or mute the thread https://github.com/notifications/unsubscribe-auth/AM5hEzKtZHluoY6HqiEoYxao8_04O0EWks5rQoQugaJpZM4LdgjR .