Closed farovictor closed 9 months ago
I have batched 1k request against your target, with no issue.
I have added better support to catch the text representation of this faulty binary issue https://github.com/scrapfly/python-scrapfly/commit/712b37a6ec843d51d59e4154923cfdaa52664337#diff-42acd60fcbec2f0f0da4cdc1f17124c2696319b3868b29b7026c76b25d675dfeR111, If you get it, you can share me the base64.
If you can, share a minimum setup to get the same condition as yours (with a poetry lock or requirement.txt with a fixed version). Because in your stack trace, you have /twisted/internet/
, I guess there is scrapy involved? (I tested regular SDK and Scrapy integration)
We have moved to a more up-to-date version of Scrapy and since we have no occurrences of this. I will close this issue.
I'm having the following issue when scraping yielding ScrapflyScrapyRequest:
This error is present in 2% of my total requests and it's completely random some URLs may hit this error in a few tries, but in most cases, they don't repeat.
Environment Setup: