Job processing freezes when job result is the job itself

[ebekebe]

Consider the following:

jobQueue.process((job, next) => {
  next(null, job)
})

This kept the job waiting forever and blocked the queue. No error is thrown and it took me several hours to track this down in a bigger project.

Obviously, it's not a good idea to pass the job itself as a result. However, in our project this was obfuscated through a couple of abstraction layers, so it would be nice to have rethinkdb-job-queue throw an error on this occasion instead of silently freezing.

[grantcarthew]

Hi @ebekebe,

As documented here and here the ability to pass the job back to the next() callback is an advanced feature and requires job property changes prior to calling next(null, job).

The above code in your comment is non-functional.

This kept the job waiting

This is correct, the job will never complete and will always be placed into a waiting state.

forever blocked the queue

This can't happen. If it is happening then there is a bug. The whole queue will not block because one job is stuck in a waiting state. That being said, if your process function is similar to your comment above then every job will be waiting forever.

Please have a read of the two documents above and get back to me if I haven't understood your issue or you are still having problems.

[ebekebe]

Thanks, that explains the behavior. I didn't know about the advanced feature.

I still think it is a hard to debug case, because I couldn't find any hint in the job's log that suggested it was updated or a timeout occurred. The job only went to the "failed" status without any notice why it failed.

[grantcarthew]

@ebekebe When you pass a job back into the next() callback internally it is calling the job-update.js module. Here it is for reference: https://github.com/grantcarthew/node-rethinkdb-job-queue/blob/master/src/job-update.js

This means there is a Job Updated log entry against the job. I agree this is not ideal.

I am going to reopen this to add a new log entry when the job is returned in the next() callback.

[grantcarthew]

Updated in master branch. When the next(null, job) callback passes the job back a new log entry is added to the Job.log array with the message "Job has not been processed to completion and is being placed back into the queue".

Published with version v3.1.1.