search

scrapinghub / splash

Lightweight, scriptable browser as a service with an HTTP API
BSD 3-Clause "New" or "Revised" License
4.07k stars 513 forks source link

ScriptError -> LUA_ERROR #763

Open emy-lee opened 6 years ago

emy-lee commented 6 years ago

HTTP Error 400 (Bad Request) Type: ScriptError -> LUA_ERROR

Error happened while executing Lua script

Lua error: [string "function main(splash, args) ..."]:2: http404

{
    "info": {
        "type": "LUA_ERROR",
        "line_number": 2,
        "source": "[string \"function main(splash, args)\r...\"]",
        "error": "http404",
        "message": "Lua error: [string \"function main(splash, args)\r...\"]:2: http404"
    },
    "type": "ScriptError",
    "description": "Error happened while executing Lua script",
    "error": 400
}
# pip list
adium-theme-ubuntu (0.3.4)
apsw (3.8.11.1.post1)
attrs (15.2.0)
BeautifulSoup (3.2.1)
beautifulsoup4 (4.4.1)
boto (2.38.0)
chardet (2.3.0)
CherryPy (3.5.0)
cryptography (1.2.3)
cssselect (0.9.1)
cssutils (1.0)
decorator (4.0.6)
deluge (1.3.15)
Django (1.8.7)
dnspython (1.12.0)
enum34 (1.1.2)
feedparser (5.1.3)
html5lib (0.999)
idna (2.0)
ipaddress (1.0.16)
ipython (2.4.1)
lxml (3.5.0)
Mako (1.0.3)
Markdown (2.6.6)
MarkupSafe (0.23)
mechanize (0.2.5)
mysqlclient (1.3.7)
ndg-httpsclient (0.4.0)
netifaces (0.10.4)
numpy (1.11.0)
PAM (0.4.2)
pexpect (4.0.1)
Pillow (3.1.2)
pip (8.1.1)
ptyprocess (0.5)
pyasn1 (0.1.9)
pyasn1-modules (0.0.7)
pycrypto (2.6.1)
pycurl (7.43.0)
pygame (1.9.1release)
Pygments (2.1)
pygobject (3.20.0)
pyOpenSSL (0.15.1)
pyparsing (2.0.3)
pyserial (3.0.1)
python-apt (1.1.0b1+ubuntu0.16.4.1)
python-dateutil (2.4.2)
python-libtorrent (1.0.11)
pytz (2014.10)
pyxdg (0.25)
PyYAML (3.11)
queuelib (1.1.1)
repoze.lru (0.6)
requests (2.9.1)
Routes (2.2)
Scrapy (1.0.3)
scrapy-splash (0.7.2)
service-identity (16.0.0)
setproctitle (1.1.8)
setuptools (20.7.0)
simplegeneric (0.8.1)
six (1.10.0)
sqlparse (0.1.18)
Twisted (16.0.0)
unity-lens-photos (1.0)
urllib3 (1.13.1)
uTidylib (0.2)
w3lib (1.11.0)
WebOb (1.5.1)
wheel (0.29.0)
zope.interface (4.1.3)
You are using pip version 8.1.1, however version 10.0.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

This is the webpage that I have to crawl: http://www.oddsportal.com/soccer/italy/serie-a-2016-2017/results/

# docker run -p 5023:5023 -p 8050:8050 -p 8051:8051 scrapinghub/splash
2018-04-19 12:14:38+0000 [-] Log opened.
2018-04-19 12:14:38.956897 [-] Splash version: 3.2
2018-04-19 12:14:39.127226 [-] Qt 5.9.1, PyQt 5.9, WebKit 602.1, sip 4.19.3, Twisted 16.1.1, Lua 5.2
2018-04-19 12:14:39.127453 [-] Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609]
2018-04-19 12:14:39.127635 [-] Open files limit: 1048576
2018-04-19 12:14:39.127782 [-] Can't bump open files limit
2018-04-19 12:14:39.427857 [-] Xvfb is started: ['Xvfb', ':1904342225', '-screen', '0', '1024x768x24', '-nolisten', 'tcp']
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2018-04-19 12:14:48.901648 [-] proxy profiles support is enabled, proxy profiles path: /etc/splash/proxy-profiles
2018-04-19 12:14:51.767429 [-] verbosity=1
2018-04-19 12:14:51.767754 [-] slots=50
2018-04-19 12:14:51.768084 [-] argument_cache_max_entries=500
2018-04-19 12:14:51.768763 [-] Web UI: enabled, Lua: enabled (sandbox: enabled)
2018-04-19 12:14:51.768911 [-] Server listening on 0.0.0.0:8050
2018-04-19 12:14:51.769866 [-] Site starting on 8050
2018-04-19 12:14:51.770000 [-] Starting factory <twisted.web.server.Site object at 0x7f2b11b157f0>
2018-04-19 12:15:51.121804 [-] "172.17.0.1" - - [19/Apr/2018:12:15:50 +0000] "GET / HTTP/1.1" 200 7675 "http://localhost:8050/info?wait=0.5&images=1&expand=1&timeout=90.0&url=http%3A%2F%2Fwww.oddsportal.com%2Fsoccer%2Fitaly%2Fserie-a-2016-2017%2Fresults%2F%23%2Fpage%2F2%2F&lua_source=function+main%28splash%2C+args%29%0D%0A++assert%28splash%3Ago%28args.url%29%29%0D%0A++assert%28splash%3Await%280.5%29%29%0D%0A++return+%7B%0D%0A++++html+%3D+splash%3Ahtml%28%29%2C%0D%0A++++png+%3D+splash%3Apng%28%29%2C%0D%0A++++har+%3D+splash%3Ahar%28%29%2C%0D%0A++%7D%0D%0Aend" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
2018-04-19 12:17:05.425987 [-] "172.17.0.1" - - [19/Apr/2018:12:17:04 +0000] "GET /info?wait=0.5&images=1&expand=1&timeout=90.0&url=http%3A%2F%2Fwww.oddsportal.com%2Fsoccer%2Fitaly%2Fserie-a-2016-2017%2Fresults%2F&lua_source=function+main%28splash%2C+args%29%0D%0A++assert%28splash%3Ago%28args.url%29%29%0D%0A++assert%28splash%3Await%280.5%29%29%0D%0A++return+%7B%0D%0A++++html+%3D+splash%3Ahtml%28%29%2C%0D%0A++++png+%3D+splash%3Apng%28%29%2C%0D%0A++++har+%3D+splash%3Ahar%28%29%2C%0D%0A++%7D%0D%0Aend HTTP/1.1" 200 5638 "http://localhost:8050/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
process 1: D-Bus library appears to be incorrectly set up; failed to read machine uuid: UUID file '/etc/machine-id' should contain a hex string of length 32, not length 0, with no other text
See the manual page for dbus-uuidgen to correct this issue.
qt.network.ssl: QSslSocket: cannot resolve SSLv2_client_method
qt.network.ssl: QSslSocket: cannot resolve SSLv2_server_method
2018-04-19 12:17:20.692460 [events] {"client_ip": "172.17.0.1", "status_code": 400, "method": "POST", "active": 0, "args": {"html5_media": false, "viewport": "1024x768", "har": 1, "save_args": [], "http_method": "GET", "images": 1, "render_all": false, "resource_timeout": 0, "html": 1, "url": "http://www.oddsportal.com/soccer/italy/serie-a-2016-2017/results/", "wait": 0.5, "load_args": {}, "uid": 139822957188880, "png": 1, "lua_source": "function main(splash, args)\r\n  assert(splash:go(args.url))\r\n  assert(splash:wait(0.5))\r\n  return {\r\n    html = splash:html(),\r\n    png = splash:png(),\r\n    har = splash:har(),\r\n  }\r\nend", "timeout": 90, "response_body": false}, "load": [2.49, 2.46, 1.77], "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36", "error": {"type": "ScriptError", "description": "Error happened while executing Lua script", "error": 400, "info": {"type": "LUA_ERROR", "message": "Lua error: [string \"function main(splash, args)\r...\"]:2: http404", "line_number": 2, "error": "http404", "source": "[string \"function main(splash, args)\r...\"]"}}, "timestamp": 1524140240, "qsize": 0, "path": "/execute", "rendertime": 12.809690713882446, "_id": 139822957188880, "fds": 20, "maxrss": 79072}
2018-04-19 12:17:20.692888 [-] "172.17.0.1" - - [19/Apr/2018:12:17:19 +0000] "POST /execute HTTP/1.1" 400 307 "http://localhost:8050/info?wait=0.5&images=1&expand=1&timeout=90.0&url=http%3A%2F%2Fwww.oddsportal.com%2Fsoccer%2Fitaly%2Fserie-a-2016-2017%2Fresults%2F&lua_source=function+main%28splash%2C+args%29%0D%0A++assert%28splash%3Ago%28args.url%29%29%0D%0A++assert%28splash%3Await%280.5%29%29%0D%0A++return+%7B%0D%0A++++html+%3D+splash%3Ahtml%28%29%2C%0D%0A++++png+%3D+splash%3Apng%28%29%2C%0D%0A++++har+%3D+splash%3Ahar%28%29%2C%0D%0A++%7D%0D%0Aend" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
# lua -v
The program 'mvn' can be found in the following packages:
 * lua5.2
 * lua5.1
 * lua50
Try: sudo apt-get install <selected package>

I will appreciate any help from you.

Gallaecio commented 4 years ago

Did you ever manage to solve your issue?

kamrankausar commented 3 years ago

{ "error": 400, "type": "ScriptError", "description": "Error happened while executing Lua script", "info": { "source": "[string \"function main(splash, args)\r...\"]", "line_number": 2, "error": "network1", "type": "LUA_ERROR", "message": "Lua error: [string \"function main(splash, args)\r...\"]:2: network1" } }