istresearch / scrapy-cluster

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
http://scrapy-cluster.readthedocs.io/
MIT License
1.18k stars 323 forks source link

adding connection timeouts for redis clients #207

Closed ronniesworld closed 5 years ago

ronniesworld commented 5 years ago

Adding socket connect and connection timeouts to the Redis client to prevent connections from hanging indefinitely.

coveralls commented 5 years ago

Coverage Status

Coverage increased (+0.03%) to 70.992% when pulling 5db88cbb55e330a53c451c54b3796e1d7fbf40bc on add-redis-socket-timeout into 55b784d1220927ef628be9b1cb0452b1cb4e247c on dev.

ronniesworld commented 5 years ago

@madisonb I updated the changelog to reflect the changes to the rest module as well.