Open linukey opened 6 years ago
the 'Bad request to splash' error maybe caused by 'local last_response = entries[#entries].response', but i don't konw how to fix it.
I have a similar issue. For some requests which I make, splash:history()
returns an empty array, which makes subsequent indexing into entries[#entries]
throw an error. What could cause Splash to not populate the history? And how to get resulting headers and http status in this case?
Yeah, it can be the problem. It is caused by cache: when response is fetched from an in-memory cache, it doesn't get a record in splash:history. I don't have a good workaround now; it makes sense to check if history is not empty before taking last entry.
@kmike I am fine to disable cache (in fact, I would prefer to do that). It seems like it's not possible until https://github.com/scrapinghub/splash/pull/339 is merged? Related issues: https://github.com/scrapinghub/splash/issues/203, https://github.com/scrapinghub/splash/issues/519.
hi kmike, i use scrapy-splash and meet a issue, when i first run 'scrapy crawl toutiao', it's run right, bug when i run it's second, it occur a issue.
i find the issue because headers i add, when i not use headers, it's run right, but it's errors when i use headers and run the second.
the lua script and project follows, i need your help, thanks.
code:
the first run correct:
the second run error: