ulixee / hero

The web browser built for scraping
MIT License
977 stars 46 forks source link

Chrome >121 does not seem to work #293

Closed NN-Binary closed 3 months ago

NN-Binary commented 3 months ago

I have tried re-installing everything, clean directory too, run as root or not, I always get this error (linux).

2024-08-28T09:38:17.441Z ERROR [hero-core/connections/ConnectionToHeroClient] ConnectionToClient.HandleRequestError { context: {}, sessionId: undefined, sessionName: undefined } Error: Failed to launch chrome 125.0.6422.141:

Please re-install the browser engine:
-------------------------------------------------
-------------- NPM INSTALL ----------------------
-------------------------------------------------

 npm install @ulixee/chrome-125-0

-------------------------------------------------

    at BrowserEngine.verifyLaunchable (/home/user/hero/program/plugins/default-browser-emulator/lib/BrowserEngine.ts:56:13)
    at Browser.launch (/home/user/hero/program/agent/main/lib/Browser.ts:136:43)
    at Object.cb (/home/user/hero/program/agent/main/lib/Pool.ts:179:21)
    at Queue.next (/home/user/hero/program/node_modules/commons/lib/Queue.ts:188:44)
    at Queue.run (/home/user/hero/program/node_modules/commons/lib/Queue.ts:72:10)
    at Pool.getBrowser (/home/user/hero/program/agent/main/lib/Pool.ts:143:44)
    at Agent.open (/home/user/hero/program/agent/main/lib/Agent.ts:116:34)
    at async Session.openBrowser (/home/user/hero/program/node_modules/core/lib/Session.ts:276:5)
    at async Function.create (/home/user/hero/program/node_modules/core/lib/Session.ts:957:7)
    at async ConnectionToHeroClient.createSession (/home/user/hero/program/node_modules/core/connections/ConnectionToHeroClient.ts:196:30)
------BROWSER_CREATION_Q--------------------------
    at Queue.run (/home/user/hero/program/node_modules/commons/lib/Queue.ts:63:19)
    at Pool.getBrowser (/home/user/hero/program/agent/main/lib/Pool.ts:143:44)
    at Agent.open (/home/user/hero/program/agent/main/lib/Agent.ts:116:34)
    at async Session.openBrowser (/home/user/hero/program/node_modules/core/lib/Session.ts:276:5)
    at async Function.create (/home/user/hero/program/node_modules/core/lib/Session.ts:957:7)
    at async ConnectionToHeroClient.createSession (/home/user/hero/program/node_modules/core/connections/ConnectionToHeroClient.ts:196:30)
    at async CommandRunner.runFn (/home/user/hero/program/node_modules/core/lib/CommandRunner.ts:36:14)
    at async ConnectionToHeroClient.executeCommand (/home/user/hero/program/node_modules/core/connections/ConnectionToHeroClient.ts:258:12)
    at async ConnectionToHeroClient.handleRequest (/home/user/hero/program/node_modules/core/connections/ConnectionToHeroClient.ts:66:14)
Automatically shutting down Hero Core (Browser Closed)
2024-08-28T09:38:17.449Z ERROR [hero-core/index] UnhandledRejection { context: {}, sessionId: null, sessionName: undefined } Error: The socket was closed while data was being compressed
    at /home/user/hero/program/node_modules/ws/lib/sender.js:510:21
    at /home/user/hero/program/node_modules/ws/lib/permessage-deflate.js:326:9
    at PerMessageDeflate.cleanup (/home/user/hero/program/node_modules/ws/lib/permessage-deflate.js:143:9)
    at WebSocket.emitClose (/home/user/hero/program/node_modules/ws/lib/websocket.js:267:57)
    at Receiver.receiverOnFinish (/home/user/hero/program/node_modules/ws/lib/websocket.js:1209:20)
    at Receiver.emit (node:events:520:28)
    at finish (node:internal/streams/writable:953:10)
    at node:internal/streams/writable:934:13
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
DisconnectedFromCore: This transport has been disconnected (host: This Hero has been disconnected from Core (coreHost: ws://localhost:1818/hero))
    at ConnectionToHeroCore.beforeDisconnect (/home/user/hero/program/node_modules/client/connections/ConnectionToHeroCore.ts:115:11)
    at ConnectionToHeroCore.onConnectionTerminated (/home/user/hero/program/node_modules/net/lib/ConnectionToCore.ts:243:42)
    at WsTransportToCore.finalHandler (/home/user/hero/program/node_modules/commons/lib/EventSubscriber.ts:46:14)
    at Object.onceWrapper (node:events:635:26)
    at WsTransportToCore.emit (node:events:520:28)
    at WsTransportToCore.emit (/home/user/hero/program/node_modules/commons/lib/TypedEventEmitter.ts:158:18)
    at WsTransportToCore.disconnect (/home/user/hero/program/node_modules/net/lib/WsTransportToCore.ts:60:10)
    at WebSocket.finalHandler (/home/user/hero/program/node_modules/commons/lib/EventSubscriber.ts:46:14)
    at Object.onceWrapper (node:events:635:26)
    at WebSocket.emit (node:events:520:28)
------AGENT QUEUE---------------------------------
    at Queue.run (/home/user/hero/program/node_modules/commons/lib/Queue.ts:63:19)
    at CoreSessions.create (/home/user/hero/program/node_modules/client/lib/CoreSessions.ts:34:8)
    at ConnectionToHeroCore.createSession (/home/user/hero/program/node_modules/client/connections/ConnectionToHeroCore.ts:67:38)
    at Hero._Hero_getCoreSessionOrReject (/home/user/hero/program/node_modules/client/lib/Hero.ts:686:10)
    at Hero.get activeTab [as activeTab] (/home/user/hero/program/node_modules/client/lib/Hero.ts:176:33)
    at Hero.goto (/home/user/hero/program/node_modules/client/lib/Hero.ts:477:17)
    at SpawnBrowser (/home/user/hero/program/browse.js:1807:32)
    at async ProcessMessage (/home/user/hero/program/browse.js:3487:26)
    at async MessagePort.<anonymous> (/home/user/hero/program/browse.js:3659:18) {
  host: 'This Hero has been disconnected from Core (coreHost: ws://localhost:1818/hero)',
  code: 'DisconnectedFromCore',
  coreHost: 'ws://localhost:1818/hero'
}
RESULT {
  error: true,
  msg: 'DisconnectedFromCore: This transport has been disconnected (host: This Hero has been disconnected from Core (coreHost: ws://localhost:1818/hero))',
  stats: { start: 1724837896594, end: 1724837897451, ms: 857 }
}

packages.json


{
    "scripts": {
        "start": "sudo bash ./bash/chrome_sandbox.sh && node engine.js"
    },
    "dependencies": {
        "@ulixee/chrome-121-0": "^6167.185.8",
        "@ulixee/chrome-122-0": "^6261.129.11",
        "@ulixee/chrome-123-0": "^6312.124.11",
        "@ulixee/chrome-124-0": "^6367.208.11",
        "@ulixee/chrome-125-0": "^6422.142.11",
        "@ulixee/chrome-126-0": "^6478.183.11",
        "@ulixee/chrome-127-0": "^6533.120.11",
        "@ulixee/cloud": "^2.0.0-alpha.29",
        "@ulixee/execute-js-plugin": "^2.0.0-alpha.29",
        "@ulixee/hero": "^2.0.0-alpha.29",
        "@ulixee/hero-core": "^2.0.0-alpha.29",
        "@ulixee/hero-playground": "^2.0.0-alpha.29",
        "@ulixee/hero-plugin-utils": "^2.0.0-alpha.29",
        "@ulixee/net": "^2.0.0-alpha.29",
        "@ulixee/unblocked-specification": "^2.0.0-alpha.29",
        "acorn": "^8.12.0",
        "acorn-walk": "^8.3.3",
        "async-mutex": "^0.5.0",
        "axios": "^1.7.2",
        "better-sqlite3": "^11.1.2",
        "buffer-image-size": "^0.6.4",
        "cheerio": "^1.0.0-rc.12",
        "express": "^4.19.2",
        "expressjs": "^1.0.1",
        "form-data": "^4.0.0",
        "node-fetch": "^3.3.2",
        "node-schedule": "^2.1.1",
        "openai": "^4.51.0",
        "rxdb": "^15.24.0",
        "rxdb-premium": "^15.24.0",
        "rxjs": "^7.8.1",
        "sharp": "^0.33.4",
        "watch": "^1.0.2"
    },
    "accessTokens": {
    }
}
NN-Binary commented 3 months ago

How do we repopulate ~/.cache/ulixee/?

blakebyrnes commented 3 months ago

Did it install your npm modules under root? Maybe has to do with how you're running npm install? Who is ~ above?

DynamicAdsAmateur commented 3 months ago

same here m1 node v20.5.0 chrome 128 (latest) installed playground and cloud under sudo, ran cloud with sudo Cannot even run basic example. Immediately after running script I see chrome popup saying it quit unexpectedly.

// sorry, not the same error, mine says "Failed to launch Chrome! Cancel Pending Promise (Target.setAutoAttach): Target closed.\n"

blakebyrnes commented 3 months ago

same here

m1

node v20.5.0

chrome 128 (latest)

installed playground and cloud under sudo, ran cloud with sudo

Cannot even run basic example. Immediately after running script I see chrome popup saying it quit unexpectedly.

// sorry, not the same error, mine says "Failed to launch Chrome! Cancel Pending Promise (Target.setAutoAttach): Target closed.\n"

Sorry you're having issues too. I think this is actually a different issue. Would you mind moving this to a different ticket and giving me a fuller set of logs before and after the error and your basic project setup? Thanks!

NN-Binary commented 3 months ago

Ok! It does seem that it's fine after completely re-installing everything, it's really strange as this happened right after installing the latest Chrome from Ulixee, but it's now solved.

@blakebyrnes do you mind telling me in new Hero() how to specify the /tmp directory, it uses /tmp/.ulixee, I can't seem to find anywhere how to change it, I want each instance to use a different dir.

blakebyrnes commented 3 months ago

Ok, I guess that's good?? Haha. Kind of hard to know what went wrong there. Maybe you grabbed chrome 128 before the data files were ready or something?

You can change tmp dir per core: https://ulixee.org/docs/hero/overview/configuration#data-dir

NN-Binary commented 3 months ago

Thank you so much for the help, I'm really unsure what happened, really strange, it was all working well.