msavin / SteveJobs

A simple jobs queue that just works (for Meteor.js)
Other
207 stars 35 forks source link

Job did not run until next job queued #68

Closed vuhrmeister closed 5 years ago

vuhrmeister commented 6 years ago

I just faced a strange behavior:

A job which should run immediately (and always did) just did not run. After half an hour I restarted the server without effect, the job still did not run. I needed to queue a new job which caused both jobs to be run.

Honestly, the job even did not fail or something. This is the document before it ran:

{
    "_id" : "eh7mg76khmBaaYD3p",
    "name" : "myJob",
    "created" : ISODate("2018-10-15T20:45:58.042+02:00"),
    "serverId" : "DWuBACmLximRk4Yo7",
    "state" : "pending",
    "due" : ISODate("2018-10-15T20:45:58.042+02:00"),
    "priority" : NumberInt("0"),
    "arguments" : [
        "Mi36bxpGoNbFzAdLx"
    ],
}

Also there was nothing in the server logs nor in Meteor APM. The jobs_dominator lastPing was up to date.

So sorry, this problem description is pretty vague, but maybe you (or someone else) faced that or a similar issue already.

I'm using version 3.0.5

msavin commented 5 years ago

@vuhrmeister I had not faced this kind of issue. Maybe a timezone issue on the server?

vuhrmeister commented 5 years ago

Can't imagine that. The timezone is properly set.

msavin commented 5 years ago

Can you post all your Jobs-related code

On Thu, Nov 1, 2018 at 12:14 PM Valentin Uhrmeister < notifications@github.com> wrote:

Can't imagine that. The timezone is properly set.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/msavin/SteveJobs..meteor.jobs.scheduler.queue.background.tasks/issues/68#issuecomment-435009398, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7-g-gb45u6ub1vgd-TCW5d2GlvR3OCks5uqteSgaJpZM4Xc_Ve .

vuhrmeister commented 5 years ago

Well, that's hard to do. First, it's a lot. Second, it's internal business logic.

But basically it's just registering the job and running without any configuration (simplified, but still the original structure):

Jobs.register({
  'myJob' (docId) {
    const doc = Docs.findOne({
      _id: docId,
      status: Status.CREATED,
    })

    if (doc) {
      let $set = {}

      if (someVar.length > 0) {
        $set = {
          status: Status.MATCHED,
        }
      } else {
        $set = {
          status: Status.NOT_MATCHED,
        }
      }

      Docs.update(docId, { $set })
    }

    this.success()
  }
})

// Another file
Jobs.run('myJob', docId)

So as the code in the original post shows triggering the job was properly done. It just got never executed. Only with the next queued job it was executed. So I don't see any reason why the actual job code should be the issue.

vuhrmeister commented 5 years ago

This issue just happened again. But this time obviously no server restart nor queuing a new job helped. I then changed the due field to be in the future. A few seconds after due date this and the older job run.

msavin commented 5 years ago

I think I got it. Probably, the queue is stopping due to a code error. Whenever the queue hits a code error, it stops running jobs until a restart (in an assumption that the code error would be fixed on that restart). If you have access to your server logs, you should be able to spot it.

The issue might be around the someVar inside your if statement, it looks undefined?

msavin commented 5 years ago

Closing on the assumption that you had moved past this issue. If anything comes up, please let me know.

vuhrmeister commented 5 years ago

Even though I see the server logs, I don't see any application related error which could be causing this. It happens so rarely that it's hard to debug it. It didn't happen for quite a while.