Closed brunomperes closed 7 years ago
@brunomperes just install scrapy-fake-useragent
with pip
:
pip install scrapy-fake-useragent
And then configure your Scrapy settings to use the middleware.
Thanks for the interest in the project.
Thanks for the PR.
I'm a bit confused, I've never used setuptools, how do I add this to a scrapy project?
I've found this post (http://www.bertcarremans.be/avoiding-ip-banning-with-the-scrapy-framework/) but it doesn't seem the most up to date way of doing so.
Thanks!