issues
search
apify
/
actor-templates
This project is the :house: home of Apify actor template projects to help users quickly get started.
https://apify.com/
24
stars
14
forks
source link
Scrapy Actor: Requests sometimes remained stuck in the RQ
#259
Closed
vdusek
closed
8 months ago
vdusek
commented
8 months ago
Scrapy Actor: Requests sometimes remained stuck in the RQ.
Which means Scrapy crawling process is stuck in the infinity loop.
I did find out why this is happening - the problem is in the
ApifyRetryMiddleware.process_exception()
.
See also this thread:
https://discord.com/channels/801163717915574323/1180618732725338182