Open starrify opened 8 years ago
Hey @starrify,
I've seen problems with redirects; they sometimes affect Splash testing suite. For some reason it is also a much larger issue on OS X than in Ubuntu (or in Docker).
I'm not sure how to fix that; it looks like a bug somewhere deep in qt/qtwebkit/webkit. We can add qtwebengine support (https://github.com/scrapinghub/splash/issues/349) or use a qtwebkit fork with updated engine (https://github.com/annulen/webkit), maybe this can help. But it is a huge amount of work.
Issue
Sometimes meta refresh redirection is not performed, even after a 10 second
wait
.Affected versions
It's confirmed to be reproducible in version 2.1, 2.2, and 2082c4b which is the latest commit at this moment.
Steps to reproduce, and other helpful info
Here is a sample page to help reproducing the issue:
In the test below, the Splash service is provided by a Docker image built from 2082c4b. For the test, a new container is started and used by only the test.
This is a minimal example that could reproduce this issue. Start with this following Lua script:
And try rendering
http://libstarrify.so/rodata/test_1.html
for 100 times:2 out of those 100 runs gave positive results where the redirection did not happen:
Here is some further info that might help investigating:
Questions
Does this HAR entry mean that the request to
http://httpbin.org/get
was cached ("ok": true
and"statusText": "OK"
)? (If so, there shall be something else wrong that prevents the redirection)If otherwise the request is failed, can
splash:wait
return some info indicating that? (e.g. we may receivenil, "network2"
from a call tosplash:go
)