Update Python Scrapy Actor template to use the Request Queue as storage - Githubissues

apify / actor-templates

This project is the :house: home of Apify actor template projects to help users quickly get started.

https://apify.com/

25 stars 14 forks source link

Update Python Scrapy Actor template to use the Request Queue as storage #183

Closed vdusek closed 1 year ago

vdusek commented 1 year ago

Current behavior

Python Scrapy Actor uses Scrapy's default Scheduler with its default storage.

Expected behavior

Python Scrapy Actor uses our custom (asynchronous) Scheduler with our Request Queue as a storage.

Possible solutions

The necessary investigation was already made:
- https://github.com/apify/apify-core/issues/11884
- https://github.com/apify/apify-core/issues/12073.

Additional context

Apify SDK 1.1.4 should be used with this fix https://github.com/apify/apify-sdk-python/pull/113.