istresearch / scrapy-cluster

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
http://scrapy-cluster.readthedocs.io/
MIT License
1.17k stars 323 forks source link

Dependency Resolution Fixes #242

Closed madisonb closed 3 years ago

madisonb commented 3 years ago

Fixes the new pip dependency resolution issue that continues to plague many repos. Now the scutils library can be used much safer with external projects.

Other updates include: