gildas-lormeau / single-file-cli

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
GNU Affero General Public License v3.0
589 stars 62 forks source link

Invalid InterceptionId error when saving a web page of some domains #111

Closed prushh closed 2 weeks ago

prushh commented 1 month ago

I am running single-file-cli in a custom Docker container with node:20-alpine3.20 as base image. The chromium-swiftshader package was installed as in your Dockerfile (Chromium 127.0.6533.88 Alpine Linux instead of v124) and the ISDCAC browser extension was added.

I'm using child_process.fork() function in a TypeScript project as follows:

Show code ````typescript export async function runSingleFile(url: string, filename: string, useProxy: boolean): Promise { const singleFileScript: string = "./node_modules/single-file-cli/single-file-node.js"; const args: string[] = [ "--browser-wait-until", "networkAlmostIdle", "--browser-executable-path", process.env.CHROME_BIN || "", "--filename-template", filename, "--user-agent", getUserAgent(), "--browser-args", `["--log-file=logs.txt", "--enable-logging", "--enable-extension-activity-logging", "--headless=new", "--disable-audio-output", "--disable-extensions-except=./CookieBlocker", "--load-extension=./CookieBlocker", "--disable-setuid-sandbox", "--disable-dev-shm-usage", "--no-sandbox"]`, url ]; if (useProxy) { args.push(...[ "--http-proxy-server", process.env.PROXY_URL || "", "--http-proxy-username", process.env.PROXY_USER || "", "--http-proxy-password", process.env.PROXY_PASSWORD || "", ]); } const options: ForkOptions = {timeout: 120000}; return await new Promise((resolve) => { const singleFileProcess: ChildProcess = fork(singleFileScript, args, options); singleFileProcess.on("error", (err) => { gContext.log.info(`[ERROR] single-file-cli on ${url}: ${err}`); resolve(false); }); singleFileProcess.on("exit", (code) => { if (code !== 0) { resolve(false); } resolve(true); }); }); } ````

I have always caught the error below in Alibaba, and sometimes also in Aliexpress and YouTube.

Show runtime error ````javascript node:internal/event_target:1094 process.nextTick(() => { throw err; }); ^ Error: Invalid InterceptionId. when calling Fetch.continueRequest({"requestId":"interception-job-289.1"}) at #onMessage (file:///function/single-file-cli/node_modules/simple-cdp/mod.js:196:36) at WebSocket. (file:///function/single-file-cli/node_modules/simple-cdp/mod.js:168:83) at callListener (/function/single-file-cli/node_modules/ws/lib/event-target.js:290:14) at WebSocket.onMessage (/function/single-file-cli/node_modules/ws/lib/event-target.js:209:9) at WebSocket.emit (node:events:519:28) at Receiver.receiverOnMessage (/function/single-file-cli/node_modules/ws/lib/websocket.js:1220:20) at Receiver.emit (node:events:519:28) at Receiver.dataMessage (/function/single-file-cli/node_modules/ws/lib/receiver.js:596:14) at /function/single-file-cli/node_modules/ws/lib/receiver.js:530:12 at /function/single-file-cli/node_modules/ws/lib/permessage-deflate.js:309:9 { code: -32602 } ````

Some examples of URLs by domain:

How can I prevent this problem in a general way?

gildas-lormeau commented 1 month ago

I did a quick test but I was not able to reproduce the issue, I did not install the ISDCAC browser extension though. Does the issue happen when the extension is not installed?

Edit: I can reproduce the bug sometimes with the extension installed (and without using Docker).

prushh commented 1 month ago

@gildas-lormeau The problem occurs occasionally even without loading the extension, on Docker container.

gildas-lormeau commented 1 month ago

The bug is a bit mysterious to me but should be fixed now hopefully.

prushh commented 1 month ago

@gildas-lormeau Thank you very much, I did some tests and there was no problem!

Edit: the issue continues to occur, but less frequently

gildas-lormeau commented 4 weeks ago

@prushh Feel free to provide me an example of a failing URL which would allow me to do some tests.

prushh commented 1 week ago

@gildas-lormeau Sorry for the late reply. Thanks for the new commit, I will test with different URLs and hope everything is ok now!

gildas-lormeau commented 1 week ago

@prushh No worries, I'm curious to know if this problem still occurs though..