search

lykmapipo / kue-scheduler

A job scheduler utility for kue, backed by redis and built for node.js
246 stars 47 forks source link

Lock expiry can potentially cause job to run twice #69

Open Roam-Cooper opened 7 years ago

Roam-Cooper commented 7 years ago

In kue-scheduler 0.6.3 (and higher versions as far as I know), it seems as though the test that the key expiry listener used to determine whether it should run "_onJobKeyExpiry" for the expired key is also satisfied by lock key expiries.

The test is: queue._jobExpiryKeyValidator = new RegExp('^' + queue.options.prefix + ':scheduler:');

Which a lock key of q:scheduler:locks:unique_task_name would meet.

Roam-Cooper commented 7 years ago

I am working on a pull request to resolve this issue, I just have to look into our deployment a bit further as the root of the issue is a lock not being unlocked, and the expiry of the lock passes as a scheduler expiry because the regex allows it.

Seems in our deployment it was causing the task to run twice, because when the lock key expires and passes for a scheduler expiry, it gets a lock on the expired lock key, and that lock will expire after 1000ms and also pass for a scheduler expiry. After all is said and done the "scheduler key" (really the lock's key) is reset with the tasks interval time and the process repeats.