issues
search
groupbwt
/
scrapy-boilerplate
Scrapy project boilerplate done right
MIT License
42
stars
25
forks
source link
- feat(middleware): added middleware what allow spiders to use the proxy from list;
#120
Closed
CESSIDY
closed
1 year ago
CESSIDY
commented
1 year ago
Completed the development of ProxyRotationMiddleware - which makes it possible to use proxies from the proxy list;
Implemented logic with a separate file for proxies;
Adds an example file with proxies proxy_list.example.json;
Added a description in middelware;
Middleware works in two modes, with each request proxies are selected (1 - randomly, 2 - in order);