salimk / Rcrawler

An R web crawler and scraper
http://www.sciencedirect.com/science/article/pii/S2352711017300110
Other
348 stars 93 forks source link

IP shuffeling services integration #62

Open KnutJaegersberg opened 5 years ago

KnutJaegersberg commented 5 years ago

Hi, a nice feature would be an integration with ip shuffeling services. There are numerous, i was thinking about something like https://proxycrawl.com/ It is not hard to implement it yourself, but I feel this would be a nice feature for a crawler, i.e. think of using crawlera with scrapy.