BruceDone / awesome-crawler

A collection of awesome web crawler,spider in different languages
MIT License
6.36k stars 688 forks source link

[Suggestion] Proxy API for crawling #18

Closed TheJoin95 closed 7 years ago

TheJoin95 commented 7 years ago

Hi, great repo! So I'd like to add this website https://www.free-proxies.info It's very helpful for crawling because it's grant an API for get free or premium proxy!

There are also some filters like transparent proxy or anonymous; google proxy and more.. It's always need a proxy for crawling the web!