search

vsivsi / meteor-job-collection

A persistent and reactive job queue for Meteor, supporting distributed workers that can run anywhere.
https://atmospherejs.com/vsivsi/job-collection
Other
385 stars 68 forks source link

Risk of running several servers against one mongo instance? #199

Closed vhf closed 8 years ago

vhf commented 8 years ago

Thanks for this great, well documented package!

I'm wondering about the risk of having a job scheduled once actually run N times in the case where I have N concurrent instances of node (my meteor application) running in parallel for load balancing.

Can this happen? Should I take special precautions, if so any idea how?

Thanks!

Chugot commented 8 years ago

vsivsi touched on this a bit in https://github.com/vsivsi/meteor-job-collection/issues/188 , you shouldn't run into trouble as long as you are using 1 mongo instance because all work will queue there and each worker will grab what it can when it has available capacity, the workers won't double-up on a job. You are more likely to run into issues using a replicated mongo instance, because the job will be replicated across each instance and without writing code to manage it yourself, there will be a copy of the job at each instance meaning there are multiple copies of the job ready to run.

vhf commented 8 years ago

Perfect, thanks!