ulixee / hero

The web browser built for scraping
MIT License
652 stars 32 forks source link

Socket closed before response received #151

Closed alexio777 closed 1 year ago

alexio777 commented 1 year ago

Hello, this simple example generates an error:

import Hero from "@ulixee/hero-playground";

async function check() {
  const hero = new Hero();
  await hero.goto("https://www.google.com");
  await hero.waitForPaintingStable();
  await hero.close();
}

check();

error:

Started Ulixee server at localhost:25241
2022-09-08T10:53:33.362Z ERROR [hero-core/connections/ConnectionToHeroClient] ConnectionToClient.HandleRequestError {
  context: {},
  sessionId: 'SGyG7KzJAOobqE3P7JfFe',
  sessionName: 'default-session'
} Error: Socket closed before response received
    at HttpRequestHandler.onRequest (/home/alexio/fs/platform/node_modules/mitm/handlers/HttpRequestHandler.ts:70:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Function.onRequest (/home/alexio/fs/platform/node_modules/mitm/handlers/HttpRequestHandler.ts:316:5)
    at async MitmProxy.onHttpRequest (/home/alexio/fs/platform/node_modules/mitm/lib/MitmProxy.ts:223:7)
2022-09-08T10:53:33.366Z ERROR [hero-core/index] UnhandledErrorOrRejection { context: {}, sessionId: null, sessionName: undefined } Error: Socket closed before response received
    at HttpRequestHandler.onRequest (/home/alexio/fs/platform/node_modules/mitm/handlers/HttpRequestHandler.ts:70:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Function.onRequest (/home/alexio/fs/platform/node_modules/mitm/handlers/HttpRequestHan
(https://github.com/ulixee/hero/files/9526278/SGyG7KzJAOobqE3P7JfFe.db.zip)
dler.ts:316:5)
    at async MitmProxy.onHttpRequest (/home/alexio/fs/platform/node_modules/mitm/lib/MitmProxy.ts:223:7)
------REMOTE CORE---------------------------------
    at Function.reviver (/home/alexio/fs/platform/node_modules/commons/lib/TypeSerializer.ts:233:26)
    at JSON.parse (<anonymous>)
    at Function.parse (/home/alexio/fs/platform/node_modules/commons/lib/TypeSerializer.ts:31:17)
    at WsTransportToCore.onMessage (/home/alexio/fs/platform/node_modules/net/lib/WsTransportToCore.ts:111:36)
    at WebSocket.emit (node:events:390:28)
    at Receiver.receiverOnMessage (/home/alexio/fs/platform/node_modules/ws/lib/websocket.js:1059:20)
    at Receiver.emit (node:events:390:28)
    at Receiver.dataMessage (/home/alexio/fs/platform/node_modules/ws/lib/receiver.js:517:14)
    at /home/alexio/fs/platform/node_modules/ws/lib/receiver.js:468:23
    at /home/alexio/fs/platform/node_modules/ws/lib/permessage-deflate.js:308:9
------CONNECTION----------------------------------
    at new Resolvable (/home/alexio/fs/platform/node_modules/commons/lib/Resolvable.ts:16:18)
    at createPromise (/home/alexio/fs/platform/node_modules/commons/lib/utils.ts:168:10)
    at PendingMessages.create (/home/alexio/fs/platform/node_modules/net/lib/PendingMessages.ts:47:44)
    at ConnectionToHeroCore.sendRequest (/home/alexio/fs/platform/node_modules/net/lib/ConnectionToCore.ts:153:50)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async CoreCommandQueue.sendRequest (/home/alexio/fs/platform/node_modules/client/lib/CoreCommandQueue.ts:287:12)
    at async Object.cb (/home/alexio/fs/platform/node_modules/client/lib/CoreCommandQueue.ts:229:16)
    at async Queue.next (/home/alexio/fs/platform/node_modules/commons/lib/Queue.ts:95:19)
------CORE COMMANDS-------------------------------
    at Queue.run (/home/alexio/fs/platform/node_modules/commons/lib/Queue.ts:40:19)
    at CoreCommandQueue.run (/home/alexio/fs/platform/node_modules/client/lib/CoreCommandQueue.ts:218:8)
    at CoreTab.goto (/home/alexio/fs/platform/node_modules/client/lib/CoreTab.ts:243:36)
    at Tab.goto (/home/alexio/fs/platform/node_modules/client/lib/Tab.ts:205:36)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

--------------------------------------------------
------default-session-----------------------------
------SGyG7KzJAOobqE3P7JfFe-----------------------
--------------------------------------------------

session: SGyG7KzJAOobqE3P7JfFe.db.zip

alexio777 commented 1 year ago
2022-09-08T11:28:08.889Z WARN [/home/alexio/fs/platform/node_modules/@unblocked-web/agent/lib/BrowserProcess] chrome.stderr {
  message: '[0908/132808.889038:ERROR:cert_verify_proc_builtin.cc(681)] CertVerifyProcBuiltin for www.google.com failed:',
  context: {},
  sessionId: null,
  sessionName: undefined
}
2022-09-08T11:28:08.889Z WARN [/home/alexio/fs/platform/node_modules/@unblocked-web/agent/lib/BrowserProcess] chrome.stderr {
  message: '----- Certificate i=0 (CN=www.google.com,O=Unblocked) -----',
  context: {},
  sessionId: null,
  sessionName: undefined
}
2022-09-08T11:28:08.889Z WARN [/home/alexio/fs/platform/node_modules/@unblocked-web/agent/lib/BrowserProcess] chrome.stderr {
  message: 'ERROR: No matching issuer found',
  context: {},
  sessionId: null,
  sessionName: undefined
}
alexio777 commented 1 year ago

I'm trying to fix it, I'll post the results.

blakebyrnes commented 1 year ago

Let me know if you need any help setting up unblocked (that's likely where the fix will need to be). The short story is you clone unblocked monorepo in a folder next to ulixee monorepo. Then yarn build:all in unblocked, followed by yarn build:all in ulixee