BruceDone / awesome-crawler

A collection of awesome web crawler,spider in different languages
MIT License
6.5k stars 709 forks source link

[Suggestion] Add distributed-webcrawler to Python section #10

Closed khanhicetea closed 7 years ago

khanhicetea commented 8 years ago

Your repo is so cool, so I want to contribute my crawler repo to your awesome list.

My repo is here : https://github.com/khanhicetea/distributed-webcrawler It's built on-top Docker, Python, Redis and Gearman Job Server with a distributed architecture :)

Hope it will be helpful :)