Bogdanp / dramatiq_sqs

A Dramatiq broker that can be used with Amazon SQS.
https://dramatiq.io
Other
56 stars 21 forks source link

Is this dramatiq SQS broker thread safe? #23

Open sparrowt opened 10 months ago

sparrowt commented 10 months ago

I noticed in https://github.com/Bogdanp/dramatiq/blob/master/docs/source/troubleshooting.rst it says:

Dramatiq operations on builtin brokers are thread-safe

however the broker in this repo is not built-in and I can't see any mention of thread safety nor any locking.

As far as I can see when a dramatiq task is enqueued then the following generic call chain occurs: Actor.send > Actor.send_with_options > Broker.enqueue

Looking at SQSBroker.enqueue in this repo (which doesn't do any locking):

Given the above, is this broker potentially not thread safe, e.g. for simultaneously enqueuing from multiple threads of a web server process (such as gunicorn) to the same queue?

Bogdanp commented 2 weeks ago

Looking at your notes, I think you're probably right. I had missed that detail about boto resources. If anyone wants to fix this, I'd be happy to accept a PR.