search

istresearch / scrapy-cluster

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
http://scrapy-cluster.readthedocs.io/
MIT License
1.17k stars 323 forks source link

Any suggestion on the best way to get the crawled data out to a file? #232

Closed ahoquegh closed 4 years ago

ahoquegh commented 4 years ago

Thanks