scrapinghub / splash

Lightweight, scriptable browser as a service with an HTTP API
BSD 3-Clause "New" or "Revised" License
4.07k stars 513 forks source link

"error": "network5", HTTP Error 400 (Bad Request) #848

Open pasindu-gamarachchi opened 5 years ago

pasindu-gamarachchi commented 5 years ago

I am trying to render the html of a website and keep getting the following error on the browser.

HTTP Error 400 (Bad Request) Type: ScriptError -> LUA_ERROR Error happened while executing Lua script Lua error: [string "function main(splash, args) ..."]:3: network5

{ "description": "Error happened while executing Lua script", "type": "ScriptError", "error": 400, "info": { "type": "LUA_ERROR", "line_number": 3, "message": "Lua error: [string \"function main(splash, args)\r...\"]:3: network5", "error": "network5", "source": "[string \"function main(splash, args)\r...\"]" } }

The lua script I'm running is as follows:

function main(splash, args) splash.resource_timeout = 10.0 assert(splash:go(args.url)) assert(splash:wait(2.5)) return { html = splash:html(), png = splash:png(), har = splash:har(), } end`

This error seems to occur mainly on this site, and splash renders the html for most other sites. The following are the versions I'm using : Splash version: 3.2 Qt 5.9.1, PyQt 5.9, WebKit 602.1, sip 4.19.3, Twisted 16.1.1, Lua 5.2 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609]

Any help is appreciated.

Mideen commented 5 years ago

Could you please mention your input URL here... @pasindu-gamarachchi

pasindu-gamarachchi commented 5 years ago

https://www.lowes.com/pl/Laminate-flooring-Laminate-flooring-accessories-Flooring/753030926 , this is the url I used. Thanks.

phongtnit commented 5 years ago

I had same error with another website, please refer the following image

My env: MacOS: 10.14.2 Docker: 18.06.1-ce Splash: 3.2 via docker command: docker run -it -p 8050:8050 scrapinghub/splash --max-timeout 3600

simonsez commented 5 years ago

And the same error on this site: https://www.csb123.com/deposit-accounts/interest-rates/ - any update here?

SingleSele commented 3 years ago

This problem is caused by the assert() function in the line 2 of the Lua script, meaning that the HTTP request failed. Sadly, no more information is provided. I encountered the problem and I found out that my docker container had no network connection. Finally I used sudo docker run –network=host -it -p 8050:8050 --rm scrapinghub/splash. I don't know why it works, but it works.

SingleSele commented 3 years ago

And this problem only occurs in a few environments. I can set up splash service easily on my own computer with default settings, but i met with this error on my remote virtual machine.