ulixee / secret-agent

The web scraper that's nearly impossible to block - now called @ulixee/hero
https://secretagent.dev
MIT License
670 stars 44 forks source link

mitm-socket CertsIpcHandler.onError #262

Closed hehehai closed 3 years ago

hehehai commented 3 years ago

run example error

env: node: 14.17.1 secret-agent: ^1.4.1-alpha.5

LSB Version: core-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch Distributor ID: Ubuntu Description: Ubuntu 18.04.4 LTS Release: 18.04 Codename: bionic

network location: chinese

yarn add secret-agent success

code:

import secretAgent from "secret-agent";

const { Agent } = secretAgent;

(async () => {
  console.time("agent");
  const agent = new Agent({
    humanEmulatorId: "basic",
    showReplay: false,
  });
  await agent.goto("https://example.org");
  await agent.waitForPaintingStable();
  const title = await agent.document.title;
  await agent.close();
  console.timeEnd("agent");

  console.log("Retrieved from https://example.org", {
    title,
  });
})();

run: node main.js

err:

2021-06-23T12:53:19.148Z ERROR [/home/ss/node_modules/@secret-agent/mitm-socket/lib/CertificateGenerator] CertsIpcHandler.onError {
  error: 'Error: spawn /home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect ENOENT',
  context: {},
  sessionId: undefined,
  sessionName: undefined
} Error: spawn /home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
    at onErrorNT (internal/child_process.js:467:16)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect',
  path: '/home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect',
  spawnargs: [
    '{"storageDir":"/tmp/.secret-agent","mode":"certs","debug":false,"ipcSocketPath":"/tmp/sa-ipc-certs-fbf922f0-d421-11eb-8010-838a796da3e0.sock"}'
  ]
}
WARNING: Secret-Agent is being run under "root" user - disabling Chromium sandbox! Run under regular user to get rid of this warning.
2021-06-23T12:53:19.277Z ERROR [/home/ss/node_modules/@secret-agent/mitm-socket/lib/MitmSocketSession] ProxyIpcHandler.onError {
  error: 'Error: spawn /home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect ENOENT',
  context: { sessionId: 'fbfec840-d421-11eb-8010-838a796da3e0' },
  sessionId: 'fbfec840-d421-11eb-8010-838a796da3e0',
  sessionName: 'default-session-2'
} Error: spawn /home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
    at onErrorNT (internal/child_process.js:467:16)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect',
  path: '/home/ss/node_modules/@secret-agent/mitm-socket/lib/../dist/connect',
  spawnargs: [
    '{"rejectUnauthorized":false,"clientHelloId":"Chrome88","tcpTtl":64,"tcpWindowSize":65535,"mode":"proxy","debug":false,"ipcSocketPath":"/tmp/sa-ipc-proxy-fc0e0a80-d421-11eb-8010-838a796da3e0.sock"}'
  ]
}
blakebyrnes commented 3 years ago

I think this is because you weren't able to install the library it needs per #261.

hehehai commented 3 years ago

@blakebyrnes Sorry, these two running environments are different. I run this abnormal code normally in MAC, but the above error occurred in Ubuntu.

And from the error message, I can't confirm whether it is the running problem of dependency or the dependency is not completely installed, but there is no error in yarn install

blakebyrnes commented 3 years ago

Ah, ok. Something seems to have failed in this install too. Good point about error message - will improve that.