Open OlegYurchik opened 4 years ago
have you been able to fix this? This is kind of an issue. I don't remember seeing it until a few months ago.
I'm having this issue too.
Not sure if this is an acceptable fix for you but I have been using https://github.com/TeamHG-Memex/aquarium.
Unfortunately it's not. I've deployed the containers to aws ecs, and am using an application load balancer instead of haproxy.
I have got this error only on some pages...
I can consistently reproduce the crash on this page: https://www.steinfort.lu/news
splash is started this way: docker run -p 8050:8050 -p 5023:5023 scrapinghub/splash:3.5.0 --disable-browser-caches
I too can consistently reproduce this issue on some pages. This causes the splash to crash. Any suggestions on how to fix this issue.
I have this issue too
I turned JS off by adding "splash.js_enabled = false" before "assert(splash:go(args.url))" in lua_script. It worked just fine in my case. My lua_script: """ function main(splash, args) splash.js_enabled = false assert(splash:go(args.url)) assert(splash:wait(1)) return { html = splash:html(), png = splash:png(), har = splash:har(), } end""" (indentations don't show properly for some reason)
Well, yeah, but disabling javascript means we can also replace splash by curl and call it a day. It's not really an option.
In my case, it clearly happens on very, very bad websites loading insane amount if content. But it would still be nice to catch the exception instead of segfaulting
Well, yeah, but disabling javascript means we can also replace splash by curl and call it a day. It's not really an option.
In my case, it clearly happens on very, very bad websites loading insane amount if content. But it would still be nice to catch the exception instead of segfaulting
I've been getting the same issue, I've been using splash through docker, is there a way to handle the exception if I am using docker?
I came across this issue too and I seemed to work around it with the following (taken from https://splash.readthedocs.io/en/stable/faq.html#how-to-run-splash-in-production):
docker run -d -p 8050:8050 --restart=always scrapinghub/splash --max-timeout 3600
You will still get the error, but at least you will always have a container available to handle requests.
I tried many async requests by 15 threads to splash like that
After about twenty minutes I got this:
Can I fix it?