chilts / mongodb-queue

Message queues which uses MongoDB.
209 stars 92 forks source link

concurrency? #29

Closed coderofsalvation closed 5 years ago

coderofsalvation commented 6 years ago

Hi, i was wondering if anyone tested this package with multiple worker-processes (using multiple setIntervals/setTimeouts or cluster.fork()). Will mongo / this package prevent race-conditions?

Example: 2 process do a .get() at almost the same time..what will happen?

ronycohen commented 6 years ago

I got the same wonder.

@coderofsalvation did you make it work that way?

chilts commented 6 years ago

Yes, everything is atomically handled in MongoDB. From the readme:

## Use of MongoDB ##

Whilst using MongoDB recently and having a need for lightweight queues, I realised
that the atomic operations that MongoDB provides are ideal for this kind of job.

Since everything it atomic, it is impossible to lose messages in or around your
application. I guess MongoDB could lose them but it's a safer bet it won't compared
to your own application.

As an example of the atomic nature being used, messages stay in the same collection
and are never moved around or deleted, just a couple of fields are set, incremented
or deleted. We always use MongoDB's excellent `collection.findAndModify()` so that
each message is updated atomically inside MongoDB and we never have to fetch something,
change it and store it back.

Perhaps I should also add that you can use this library from multiple processes. I thought it was implicit, but perhaps it isn't?

crobinson42 commented 5 years ago

@chilts the readme makes it clear 👌 nice little project, btw!

chilts commented 5 years ago

@crobinson42 Thanks!

I think I'll close this now since I think we're good.