ulixee / hero

The web browser built for scraping
MIT License
652 stars 32 forks source link

Stuck at hero.goto() #132

Closed henry-bao closed 2 years ago

henry-bao commented 2 years ago

Hey, I tried the example usage. However, it looks like it's stuck at await hero.goto('https://example.org'), and the last console output is Connecting to Ulixee server at localhost:56142

Any idea of why that might happen?

Thanks

blakebyrnes commented 2 years ago

Can you paste a full example, and where you got the code from? If you're just using the "hero" package, it probably can't find a core to connect to (you can use hero-playground to start).

henry-bao commented 2 years ago

Thanks for the follow up blakebyrnes,

Right now it's giving me another error; I was running this modified example code from the repo

const Hero = require('@ulixee/hero-playground');

(async () => {
  const hero = new Hero();
  console.log('Going to a webpage...')
  await hero.goto('https://example.org');
  const title = await hero.document.title;
  const intro = await hero.document.querySelector('p').textContent;
  console.log(title, intro);
  await hero.close();
})();

and this is what came out of the console log:

Connecting to Ulixee server at localhost:57573
Going to a webpage...

node:internal/process/promises:246
          triggerUncaughtException(err, true /* fromPromise */);
          ^
DisconnectedFromCore: This transport has been disconnected (host: This Hero has been disconnected from Core (coreHost: ws://localhost:57573))
    at ConnectionToHeroCore.beforeDisconnect (/Users/henry/Desktop/untitled folder/node_modules/client/connections/ConnectionToHeroCore.ts:111:11)
    at ConnectionToHeroCore.onConnectionTerminated (/Users/henry/Desktop/untitled folder/node_modules/@ulixee/lib/ConnectionToCore.ts:236:42)
    at Object.onceWrapper (node:events:510:26)
    at WsTransportToCore.emit (node:events:390:28)
    at WsTransportToCore.emit (/Users/henry/Desktop/untitled folder/node_modules/@ulixee/net/node_modules/@ulixee/lib/eventUtils.ts:181:23)
    at WsTransportToCore.disconnect (/Users/henry/Desktop/untitled folder/node_modules/@ulixee/lib/WsTransportToCore.ts:68:10)
    at Object.onceWrapper (node:events:510:26)
    at WebSocket.emit (node:events:402:35)
    at emitErrorAndClose (/Users/henry/Desktop/untitled folder/node_modules/ws/lib/websocket.js:886:13)
    at ClientRequest.<anonymous> (/Users/henry/Desktop/untitled folder/node_modules/ws/lib/websocket.js:743:5)
------AGENT QUEUE---------------------------------
    at Queue.run (/Users/henry/Desktop/untitled folder/node_modules/@ulixee/lib/Queue.ts:40:19)
    at CoreSessions.create (/Users/henry/Desktop/untitled folder/node_modules/client/lib/CoreSessions.ts:30:8)
    at ConnectionToHeroCore.createSession (/Users/henry/Desktop/untitled folder/node_modules/client/connections/ConnectionToHeroCore.ts:63:38)
    at Hero._Hero_getCoreSessionOrReject (/Users/henry/Desktop/untitled folder/node_modules/client/lib/Hero.ts:543:10)
    at Hero.get activeTab [as activeTab] (/Users/henry/Desktop/untitled folder/node_modules/client/lib/Hero.ts:155:33)
    at Hero.goto (/Users/henry/Desktop/untitled folder/node_modules/client/lib/Hero.ts:382:17)
    at /Users/henry/Desktop/untitled folder/index.js:6:14
    at Object.<anonymous> (/Users/henry/Desktop/untitled folder/index.js:11:3)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) {
  host: 'This Hero has been disconnected from Core (coreHost: ws://localhost:57573)',
  code: 'DisconnectedFromCore',
  coreHost: 'ws://localhost:57573'
}

I am running it on an M1 Macbook macOS Monterey 12.4, node version v16.13.2

blakebyrnes commented 2 years ago

That error has been rephrased in dev to be more descriptive. It’s not actually finding any open server. Ulixee server creates a config in ~/Library/Cache/ulixee called server.json. After you close your example process, is something staying in here? Or a node process stating alive?

On Jun 26, 2022, at 4:46 AM, Henry Bao @.***> wrote:

 Thanks for the follow up blakebyrnes,

Right now it's giving me another error; I was running this modified example code from the repo

const Hero = @.***/hero-playground');

(async () => { const hero = new Hero(); console.log('Going to a webpage...') await hero.goto('https://example.org'); const title = await hero.document.title; const intro = await hero.document.querySelector('p').textContent; console.log(title, intro); await hero.close(); })(); and this is what came out of the console log:

Connecting to Ulixee server at localhost:57573 Going to a webpage...

node:internal/process/promises:246 triggerUncaughtException(err, true / fromPromise /); ^ DisconnectedFromCore: This transport has been disconnected (host: This Hero has been disconnected from Core (coreHost: ws://localhost:57573)) at ConnectionToHeroCore.beforeDisconnect (/Users/henry/Desktop/untitled folder/node_modules/client/connections/ConnectionToHeroCore.ts:111:11) at ConnectionToHeroCore.onConnectionTerminated (/Users/henry/Desktop/untitled @./lib/ConnectionToCore.ts:236:42) at Object.onceWrapper (node:events:510:26) at WsTransportToCore.emit (node:events:390:28) at WsTransportToCore.emit (/Users/henry/Desktop/untitled @*.**@*./lib/eventUtils.ts:181:23) at WsTransportToCore.disconnect (/Users/henry/Desktop/untitled @./lib/WsTransportToCore.ts:68:10) at Object.onceWrapper (node:events:510:26) at WebSocket.emit (node:events:402:35) at emitErrorAndClose (/Users/henry/Desktop/untitled folder/node_modules/ws/lib/websocket.js:886:13) at ClientRequest. (/Users/henry/Desktop/untitled folder/node_modules/ws/lib/websocket.js:743:5) ------AGENT QUEUE--------------------------------- at Queue.run (/Users/henry/Desktop/untitled @.***/lib/Queue.ts:40:19) at CoreSessions.create (/Users/henry/Desktop/untitled folder/node_modules/client/lib/CoreSessions.ts:30:8) at ConnectionToHeroCore.createSession (/Users/henry/Desktop/untitled folder/node_modules/client/connections/ConnectionToHeroCore.ts:63:38) at Hero._Hero_getCoreSessionOrReject (/Users/henry/Desktop/untitled folder/node_modules/client/lib/Hero.ts:543:10) at Hero.get activeTab [as activeTab] (/Users/henry/Desktop/untitled folder/node_modules/client/lib/Hero.ts:155:33) at Hero.goto (/Users/henry/Desktop/untitled folder/node_modules/client/lib/Hero.ts:382:17) at /Users/henry/Desktop/untitled folder/index.js:6:14 at Object. (/Users/henry/Desktop/untitled folder/index.js:11:3) at Module._compile (node:internal/modules/cjs/loader:1101:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) { host: 'This Hero has been disconnected from Core (coreHost: ws://localhost:57573)', code: 'DisconnectedFromCore', coreHost: 'ws://localhost:57573' } I am running it on an M1 Macbook macOS Monterey 12.4, node version v16.13.2

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

henry-bao commented 2 years ago

Yes, there is a server.json located in ~/Library/Caches/ulixee, and the config stays there even after I closed my example process.

blakebyrnes commented 2 years ago

Does it have anything in it? It’s fine for the file to stay.

On Jun 26, 2022, at 10:38 PM, Henry Bao @.***> wrote:

 Yes, there is a server.json located in ~/Library/Caches/ulixee, and the config stays there even after I closed my example process.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

henry-bao commented 2 years ago

Yes, the content stayed even when the process was terminated

blakebyrnes commented 2 years ago

Ok, I have a fix for this in the next version. Thanks for reporting!

henry-bao commented 2 years ago

Thank you very much, I'll close the issue now

blakebyrnes commented 1 year ago

Try removing the file and run your code again.

On Jun 27, 2022, at 8:13 AM, Henry Bao @.***> wrote:

 Yes, the content stayed even when the process was terminated

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.