search

adjust / rmq

Message queue system written in Go and backed by Redis
MIT License
1.57k stars 206 forks source link

Consider using BRPOPLPUSH #68

Closed wyxloading closed 4 years ago

wyxloading commented 5 years ago

https://github.com/adjust/rmq/blob/507bf31880496a4fd77810e0d6ea898ea2c7cc39/queue.go#L332

consume now call LLEN && RPOPLPUSH to gain queue message from redis. BRPOPLPUSH can get rid of that LLEN call, should be more efficient.

wellle commented 4 years ago

I'm changing it right now to just use RPOPLPUSH until either the delivery channel is full or the source redis list is empty (RPOPLPUSH return (nil)). That way we get rid of LLEN.

I did consider using BRPOPLPUSH but there seems to be a problem. If the queue is empty and no new deliveries get produced, if the consumer is blocked on PRPOPLPUSH and calls queue.StopConsuming() (from a different goroutine), then we don't get a chance to check the consumingStopped value inside of consume() and will remain blocked forever.

For that reason I'm closing this issue. Thanks for the suggestion though! 🙌