spulec / PyQS

Python task-queues for Amazon SQS
MIT License
174 stars 36 forks source link

Add better error handling around boto connections. #15

Open andrewgross opened 9 years ago

bdon commented 3 years ago

Hi, thanks for this library! It's been working well running on AWS for years; I recently migrated some compute-heavy jobs to a different provider however, and am observing some cases where an intermittent failure to connect to AWS causes an outage where the worker will no longer process jobs, with a stack trace like this:

 Process ReadWorker-1:
 Traceback (most recent call last):
   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
     self.run()
   File "/home/worker/venv/lib/python3.8/site-packages/pyqs/worker.py", line 88, in run
     self.read_message()
   File "/home/worker/venv/lib/python3.8/site-packages/pyqs/worker.py", line 93, in read_message
...
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL ...

Is this related to this open issue? My pyqs version is 0.1.1.