Added scrapyd, scrapyc and scrapyd-client

BruceDone / awesome-crawler

A collection of awesome web crawler,spider in different languages

MIT License

6.36k stars 688 forks source link

Closed f213 closed 6 years ago

BruceDone commented 6 years ago

Hi ,thanks for your pr ,but these are scrapy related service , not crawl framework.

f213 commented 6 years ago

Hey Bruce!

I thought it's ok, because already present stuff like Scrapy-Redis or django-dynamic-scraper are not scraping frameworks either.

May be it is ok to get full ecosystem covered in this project? Scrapy's one is not big enough to pretend for a dedicated awesome list, i think.