search

khpeek / scraper-compose

Scrapy example project using Tor (through Privoxy) in a Docker Compose multi-container application
10 stars 6 forks source link

readme

scraper-compose

Example project of anonymized scraping with Tor. (The spider is based on the Scrapy Tutorial).

Basic usage

This multi-container application uses Docker Compose. In the project directory run the command

docker-compose build

followed by

docker-compose up

This will start the Tor and Privoxy services and initiate crawling the spider.