adjust / rmq

Message queue system written in Go and backed by Redis
MIT License
1.57k stars 206 forks source link

Do Consumers work in parallel? #156

Closed kinshukb-od closed 1 year ago

kinshukb-od commented 1 year ago

Hi Greetings!!

I have one query related to consumer lifecycle From the README https://github.com/adjust/rmq#consumer-lifecycle

It has been mentioned

"For each consumer rmq takes one of the prefetched unacked deliveries from the delivery channel and passes it to the consumer's Consume() function. The next delivery will only be passed to the same consumer once the prior Consume() call returns. So each consumer will only be consuming a single delivery at any given time."

Does it imply the unacked delivery is assigned to any available consumer even while another consumer is still processing If yes can a global var being used by the Consume() process have data inconsistency?

Also, If you want to achieve parallel processing with multiple consumers using this package, you will need to implement the parallelism yourself or the basic support is inherently provided

Please reply soon Thanks

wellle commented 1 year ago

Yes every unacked delivery is assigned to a single consumer. Multiple deliveries get unacked per consumer at once, but each delivery is passed to the consumer one at a time.

Consumers do work in parallel. If you have shared state between the consumers you need to make sure it gets accessed in a thread safe way.

If you add multiple consumers they will be used in parallel automatically.

kinshukb-od commented 1 year ago

Yes every unacked delivery is assigned to a single consumer. Multiple deliveries get unacked per consumer at once, but each delivery is passed to the consumer one at a time.

Consumers do work in parallel. If you have shared state between the consumers you need to make sure it gets accessed in a thread safe way.

If you add multiple consumers they will be used in parallel automatically.

Thanks for your quick response!