Add Scrapy `ApifyHttpProxyMiddleware` for managing proxies

apify / actor-templates

This project is the :house: home of Apify actor template projects to help users quickly get started.

https://apify.com/

24 stars 14 forks source link

Add Scrapy `ApifyHttpProxyMiddleware` for managing proxies #260

Closed vdusek closed 8 months ago

vdusek commented 8 months ago

Issue

Closes: https://github.com/apify/actor-templates/issues/255

Release process

After the SDK update (https://github.com/apify/apify-sdk-python/pull/158) is merged, release a new version, and use it here.
Once we finish the review process, update the scrapy wrapper and actor-scrapy-books-example.

honzajavorek commented 7 months ago

I got it working! Awesome. My actor has been timeouting or finishing after struggling for 4min, now after implementing proxies, even the shared ones, after manual trigger it finished after 2min 💪 🎉