search

vsivsi / meteor-job-collection

A persistent and reactive job queue for Meteor, supporting distributed workers that can run anywhere.
https://atmospherejs.com/vsivsi/job-collection
Other
388 stars 68 forks source link

Retry value on jobs higher than what I set, and jobs share run ids? #244

Closed petr24 closed 7 years ago

petr24 commented 7 years ago

Quick question.

If my queue is completely empty (in the collection) and I add a job with 'retries' set to 5. The db shows that retries is set to 5. However, if I add another job with 'retries' also set to 5. The 'retry' value is now 6. And it continues to increment like that. Is this an expected result, or could be a bug?

Another question I had. if the db has a few jobs set to "ready" and then I start the queue. Some jobs share the same 'runId' even though my concurrency, payload, and prefetch are set to 1. Is that fine?

Thanks!

petr24 commented 7 years ago

Never mind. I am an idiot. I was using an object to store a set of default values. And apparently it was writing the change to that object. So I fixed the issue.

And as far as runId not sure if that's cause for concern.

vsivsi commented 7 years ago

Hi, I'm not sure I understand precisely what the remaining question is. Are you saying that you have multiple, independently created jobs that have the same runId value when they enter the "running" state?

It looks like this can happen when jobs enter the running state as a result of the same call to getWork() (which is typically called internally by processJobs()). A given run attempt for a job is uniquely identified by the combination of the jobId and the runId, so multiple jobs sharing a runId shouldn't be an issue. It's been a long time since I wrote the code that does this, but it was intentional, and necessary to efficiently convert a large batch of jobs from "ready" to "runnning" with a single MongoDB modify query.

So not a bug, and probably actually a "feature". Thanks for pointing it out though, I had forgotten about this particular behavioral quirk.

petr24 commented 7 years ago

@vsivsi ah ok, not a problem then. Was just wondering if that was intended result or not. Then we are all set. Love this package. Thank you!