issues
search
istresearch
/
scrapy-cluster
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
http://scrapy-cluster.readthedocs.io/
MIT License
1.18k
stars
324
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Disable in-memory DNS cache, add docs
#16
codywilbourn
closed
9 years ago
1
Persistent DNS cache
#15
codywilbourn
closed
9 years ago
3
Scrapy Cluster Pip Packaging
#14
madisonb
closed
8 years ago
2
Scrapy Cluster Utils Packaging
#13
madisonb
closed
8 years ago
4
Statistics Collection without 3rd Party Tools
#12
madisonb
closed
8 years ago
5
Dev
#11
sblzk
closed
9 years ago
1
Improved Debugging
#10
madisonb
closed
9 years ago
3
Modularize via Plugins/Middleware
#9
madisonb
closed
9 years ago
3
Migration Script to update from 1.0 to 1.1
#8
madisonb
closed
8 years ago
1
Scrapy Cluster 1.1 Documentation
#7
madisonb
closed
8 years ago
5
Integration Tests
#6
madisonb
closed
9 years ago
2
Redis-Monitor Unit Testing
#5
madisonb
closed
9 years ago
1
Kafka-Monitor Unit testing
#4
madisonb
closed
9 years ago
1
Cluster Speed Control
#3
madisonb
closed
9 years ago
1
Domain based queueing
#2
madisonb
closed
9 years ago
1
Scrapy 1.0 Support
#1
madisonb
closed
8 years ago
2
Previous