ulixee / hero

The web browser built for scraping
MIT License
865 stars 43 forks source link

TimeoutError: DevtoolsApiMessage did not respond after 60 seconds #294

Open dev2xl opened 2 months ago

dev2xl commented 2 months ago

I created a docker image, I am running the script and after finding the input and clicking on it I tried to type some text with await hero.type('test') and I got the following error

2024-08-31 12:43:37 ::: START :::
2024-08-31 12:43:42 2.0.0-alpha.29
2024-08-31 12:43:44 input found true
2024-08-31 12:43:44 button found true
2024-08-31 12:43:44 input clicked
2024-08-31 12:44:46 TimeoutError: DevtoolsApiMessage did not respond after 60 seconds. (Input.dispatchKeyEvent, id=117)
2024-08-31 12:44:46   at new Resolvable (/app/node_modules/commons/lib/Resolvable.ts:19:18)
2024-08-31 12:44:46     at createPromise (/app/node_modules/commons/lib/utils.ts:147:10)
2024-08-31 12:44:46     at DevtoolsSession.send (/app/agent/main/lib/DevtoolsSession.ts:82:37)
2024-08-31 12:44:46     at Keyboard.down (/app/agent/main/lib/Keyboard.ts:45:32)
2024-08-31 12:44:46     at Keyboard.press (/app/agent/main/lib/Keyboard.ts:87:16)
2024-08-31 12:44:46     at Interactor.playInteraction (/app/agent/main/lib/Interactor.ts:390:37)
2024-08-31 12:44:46     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-08-31 12:44:46     at async DefaultHumanEmulator.playInteractions (/app/plugins/default-human-emulator/index.ts:102:17)
2024-08-31 12:44:46     at async Plugins.playInteractions (/app/agent/main/lib/Plugins.ts:162:7)
2024-08-31 12:44:46     at async /app/agent/main/lib/Interactor.ts:149:18
2024-08-31 12:44:46 ------CONNECTION----------------------------------
2024-08-31 12:44:46   at new Resolvable (/app/node_modules/commons/lib/Resolvable.ts:19:18)
2024-08-31 12:44:46     at createPromise (/app/node_modules/commons/lib/utils.ts:147:10)
2024-08-31 12:44:46     at PendingMessages.create (/app/node_modules/net/lib/PendingMessages.ts:47:44)
2024-08-31 12:44:46     at ConnectionToHeroCore.sendRequest (/app/node_modules/net/lib/ConnectionToCore.ts:158:50)
2024-08-31 12:44:46     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-08-31 12:44:46     at async CoreCommandQueue.sendRequest (/app/node_modules/client/lib/CoreCommandQueue.ts:317:12)
2024-08-31 12:44:46     at async Object.cb (/app/node_modules/client/lib/CoreCommandQueue.ts:231:16)
2024-08-31 12:44:46     at async Queue.next (/app/node_modules/commons/lib/Queue.ts:188:19)
2024-08-31 12:44:46 ------CORE COMMANDS-------------------------------
2024-08-31 12:44:46     at Queue.run (/app/node_modules/commons/lib/Queue.ts:63:19)
2024-08-31 12:44:46     at CoreCommandQueue.run (/app/node_modules/client/lib/CoreCommandQueue.ts:220:8)
2024-08-31 12:44:46     at CoreFrameEnvironment.interact (/app/node_modules/client/lib/CoreFrameEnvironment.ts:135:29)
2024-08-31 12:44:46     at Function.run (/app/node_modules/client/lib/Interactor.ts:50:21)
2024-08-31 12:44:46     at Hero.type (/app/node_modules/client/lib/Hero.ts:421:22)
2024-08-31 12:44:46     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-08-31 12:44:46 
2024-08-31 12:44:46 --------------------------------------------------
2024-08-31 12:44:46 --------------------------------------------------
2024-08-31 12:44:46 ------aQW4MH4tckPqGmkXcB2mE-----------------------
2024-08-31 12:44:46 --------------------------------------------------
dev2xl commented 2 months ago

The test code

import HeroCore from '@ulixee/hero-core';
import { TransportBridge } from '@ulixee/net';
import { ConnectionToHeroCore } from '@ulixee/hero';

const bridge = new TransportBridge();
const connectionToCore = new ConnectionToHeroCore(bridge.transportToCore);

const heroCore = new HeroCore();
heroCore.addConnection(bridge.transportToClient);

import Hero from "@ulixee/hero";

(async () => {

  console.log('::: START :::')

  try {

    const hero = new Hero({
      connectionToCore,
      noChromeSandbox: true,
      sessionKeepAlive: true,
      //userAgent: "~ chrome = 128"
    });

    console.log(hero.version)

    await hero.goto("some url");
    await hero.waitForPaintingStable();

    //console.log(await hero.url);

    const input = hero.querySelector('input[placeholder="test"]');

    console.log('input found', !!input);

    const button = hero.querySelector('button');

    console.log('button found', !!button);

    await hero.click(input)

    console.log('input clicked')

    await hero.type('test')

    console.log('text typed')
  }
  catch (err) {
    console.error(err);
  }

})();
dev2xl commented 2 months ago

The package.json

{
  "name": "test",
  "version": "1.0.0",
  "scripts": {
    "dev": "ts-node src/index.ts",
    "start": "ts-node dist/index.js",
    "build": "tsc -p .",
  },
  "license": "MIT",
  "devDependencies": {
    "@types/node": "^22.5.1",
    "ts-node": "^10.9.2",
    "typescript": "^5.5.4"
  },
  "dependencies": {
    "@ulixee/chrome-128-0": "^6613.114.11",
    "@ulixee/hero": "2.0.0-alpha.29",
    "@ulixee/hero-core": "2.0.0-alpha.29",
    "@ulixee/hero-playground": "^2.0.0-alpha.29",
    "@ulixee/cloud": "^2.0.0-alpha.29"
  }
}
dev2xl commented 2 months ago

The docker image

FROM --platform=linux/amd64 node:18-slim

RUN apt-get update && \
    apt-get install -y wget gnupg && \
    rm -rf /var/lib/apt/lists/*

RUN wget --no-verbose -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor > /usr/share/keyrings/google-chrome-archive-keyring.gpg \
    && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/google-chrome-archive-keyring.gpg] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list \
    && apt-get -y update \
    && apt-get install --no-install-recommends -y google-chrome-stable xvfb xauth \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY src /app/src

COPY tsconfig.json /app/tsconfig.json

RUN npm run build

CMD [ "npm", "run", "dev"]
dev2xl commented 2 months ago

Small note, with VPN it types correctly but without it gets stuck like the error shown.

blakebyrnes commented 2 months ago

You might pull some ideas from the Dockerfile and run options in platform. I think we had to run with ipc=host to be consistent. The VPN thing is very weird

https://github.com/ulixee/platform/tree/main/cloud/tools/docker