Open danilo4pm opened 2 years ago
I'm using splash with docker compose bellow:
version: '3.7' services: splash: image: "scrapinghub/splash:3.5" restart: always container_name: "splash" ports: - "8050:8050" mem_limit: 1024m command: "--disable-private-mode -v3 --js-cross-domain-access --browser-engines=chromium"
Using python requests to query render.html with js_source as said on the documentation page :
import requests js = """document.querySelector("button[data-tracking='cc-accept']").click(); document.querySelector("div[data-testid='headerImageClickArea']").click(); document.querySelector("button.image-gallery-icon.image-gallery-right-nav").click(); """ url = 'anyurl' resp = requests.get(f'http://localhost:8050/render.html?url={url}&engine=chromium&js_source={js}') f = open("./chromium-engine.html", "w") f.write(str(resp.content)) f.close()
2022-05-11 12:35:22.020634 [events] {"path": "/render.html", "rendertime": 0.1633293628692627, "maxrss": 318320, "load": [0.31, 0.41, 0.4], "fds": 53, "active": 0, "qsize": 0, "_id": 140135661564816, "method": "GET", "timestamp": 1652272522, "user-agent": "python-requests/2.26.0", "args": {"url": "anyurl", "engine": "chromium", "js_source": "document.querySelector(\"button[data-tracking='cc-accept']\").click()", "\n document.querySelector(\"div[data-testid": "'headerImageClickArea']\").click()", "uid": 140135661564816}, "status_code": 400, "client_ip": "172.22.0.1", "error": {"error": 400, "type": "BadOption", "description": "Incorrect HTTP API arguments", "info": "js_source is not implemented"}}
I'm using splash with docker compose bellow:
Using python requests to query render.html with js_source as said on the documentation page :
2022-05-11 12:35:22.020634 [events] {"path": "/render.html", "rendertime": 0.1633293628692627, "maxrss": 318320, "load": [0.31, 0.41, 0.4], "fds": 53, "active": 0, "qsize": 0, "_id": 140135661564816, "method": "GET", "timestamp": 1652272522, "user-agent": "python-requests/2.26.0", "args": {"url": "anyurl", "engine": "chromium", "js_source": "document.querySelector(\"button[data-tracking='cc-accept']\").click()", "\n document.querySelector(\"div[data-testid": "'headerImageClickArea']\").click()", "uid": 140135661564816}, "status_code": 400, "client_ip": "172.22.0.1", "error": {"error": 400, "type": "BadOption", "description": "Incorrect HTTP API arguments", "info": "js_source is not implemented"}}