search

webdriverio / webdriverio

Next-gen browser and mobile automation test framework for Node.js
http://webdriver.io
MIT License
9.04k stars 2.5k forks source link

[🐛 Bug]: `@wdio/devtools-service` support for Selenoid #7530

Closed dmitry-lomakin closed 2 years ago

dmitry-lomakin commented 3 years ago

Environment (please complete the following information):

Config of WebdriverIO

    ...
    hostname: '10.0.x.y',
    port: 4444,
    path: '/wd/hub',
    ....
    // Test runner services
    // Services take over a specific job you don't want to take care of. They enhance
    // your test setup with almost no effort. Unlike plugins, they don't add new
    // commands. Instead, they hook themselves up into the test process.
    services: ['devtools', 'rerun'],
    ...
    capabilities: [
    {
        browserName: 'chrome',
        'selenoid:options': { enableVNC: true, enableVideo: false, hostsEntries },
    },
    ],
    ...
],

Describe the bug After the devtool service activation, error messages like this:

[0-0] 2021-10-08T07:49:55.455Z ERROR @wdio/utils:shim: SyntaxError: Unexpected token Y in JSON at position 0
[0-0]     at JSON.parse (<anonymous>)
[0-0]     at IncomingMessage.<anonymous> (/***************/e2e-wd/node_modules/lighthouse/lighthouse-core/gather/connections/cri.js:97:28)
[0-0]     at IncomingMessage.emit (events.js:412:35)
[0-0]     at IncomingMessage.emit (domain.js:470:12)
[0-0]     at endReadableNT (internal/streams/readable.js:1317:12)
[0-0]     at processTicksAndRejections (internal/process/task_queues.js:82:21)
[0-0] Error in "AfterHook Hook"
Unexpected token Y in JSON at position 0

started to appear in the output. I checked the actual request and it indeed attempts to read JSON at http://10.0.x.y:4444/json/list. The problem is that this endpoint responds with plaintext You are using Selenoid 1.10.5!.

It does not affect the execution of the tests though.

To Reproduce Steps to reproduce the behavior:

Expected behavior No error messages should appear in the logs.

Log

See an example error message above.

BorisOsipov commented 3 years ago

@dmitry-lomakin you have to add aerokube vendor capabilities 'selenoid:options': {} or 'moon:options':{} to your capabilities if you want work with selenoid and devtools features. See my examples here

dmitry-lomakin commented 3 years ago

Hi @BorisOsipov. Thank you for the reply. My config already has a 'selenoid:options' section - I've updated the issue description.

Let me try running your example against my Selenoid instance. If it passes, I'll find a diff that causes issues with my current project. I'll update this issue with the result.

dmitry-lomakin commented 3 years ago

Your example has the same issue when executed against my Selenoid instance:

(base) dmitry.lomakin@dlomakin-mb wdio-selenoid-examples % npm test

> wdio-selenoid-examples@0.0.1 test /Users/dmitry.lomakin/Sources/wdio-selenoid-examples
> wdio

Execution of 1 workers started at 2021-10-08T08:29:15.294Z

2021-10-08T08:29:15.745Z INFO @wdio/cli:launcher: Run onPrepare hook
2021-10-08T08:29:15.747Z INFO @wdio/cli:launcher: Run onWorkerStart hook
2021-10-08T08:29:15.748Z INFO @wdio/local-runner: Start worker 0-0 with arg:
[0-0] 2021-10-08T08:29:16.204Z INFO @wdio/local-runner: Run worker command: run
[0-0] RUNNING in chrome - /test/specs/example.e2e.ts
[0-0] 2021-10-08T08:29:17.044Z INFO webdriver: Initiate new session using the WebDriver protocol
[0-0] 2021-10-08T08:29:17.092Z INFO webdriver: [POST] http://10.0.X.Y:4444/wd/hub/session
[0-0] 2021-10-08T08:29:17.092Z INFO webdriver: DATA {
[0-0]   capabilities: {
[0-0]     alwaysMatch: {
[0-0]       browserName: 'chrome',
[0-0]       acceptInsecureCerts: true,
[0-0]       'selenoid:options': [Object]
[0-0]     },
[0-0]     firstMatch: [ {} ]
[0-0]   },
[0-0]   desiredCapabilities: {
[0-0]     browserName: 'chrome',
[0-0]     acceptInsecureCerts: true,
[0-0]     'selenoid:options': { enableVNC: true }
[0-0]   }
[0-0] }
[0-0] 2021-10-08T08:29:19.579Z ERROR @wdio/utils:shim: SyntaxError: Unexpected token Y in JSON at position 0
[0-0]     at JSON.parse (<anonymous>)
[0-0]     at IncomingMessage.<anonymous> (/Users/dmitry.lomakin/Sources/wdio-selenoid-examples/node_modules/lighthouse/lighthouse-core/gather/connections/cri.js:97:28)
[0-0]     at IncomingMessage.emit (events.js:412:35)
[0-0]     at IncomingMessage.emit (domain.js:470:12)
[0-0]     at endReadableNT (internal/streams/readable.js:1317:12)
[0-0]     at processTicksAndRejections (internal/process/task_queues.js:82:21)
[0-0] 2021-10-08T08:29:19.585Z INFO webdriver: COMMAND navigateTo("https://webdriver.io/")
[0-0] 2021-10-08T08:29:19.585Z INFO webdriver: [POST] http://10.0.X.Y:4444/wd/hub/session/1b42cee1833f22e9622fbe977db8369d/url
[0-0] 2021-10-08T08:29:19.585Z INFO webdriver: DATA { url: 'https://webdriver.io/' }
[0-0] {
[0-0]   Timestamp: 2589521.022378,
[0-0]   Documents: 25,
[0-0]   Frames: 19,
[0-0]   JSEventListeners: 300,
[0-0]   Nodes: 2143,
[0-0]   LayoutCount: 10,
[0-0]   RecalcStyleCount: 22,
[0-0]   LayoutDuration: 0.189904,
[0-0]   RecalcStyleDuration: 0.08845,
[0-0]   ScriptDuration: 0.430856,
[0-0]   TaskDuration: 1.390678,
[0-0]   JSHeapUsedSize: 5730368,
[0-0]   JSHeapTotalSize: 9658368
[0-0] }
[0-0] 2021-10-08T08:29:21.225Z INFO webdriver: COMMAND getWindowHandle()
[0-0] 2021-10-08T08:29:21.225Z INFO webdriver: [GET] http://10.0.X.Y:4444/wd/hub/session/1b42cee1833f22e9622fbe977db8369d/window
[0-0] 2021-10-08T08:29:21.283Z INFO webdriver: RESULT CDwindow-49C55A9B76B9BBC4BE8AAD259A4058D6
[0-0] 2021-10-08T08:29:21.454Z INFO webdriver: COMMAND navigateTo("https://todobackend.com/client/index.html?https://todo-backend-spring4-java8.herokuapp.com/todos/")
[0-0] 2021-10-08T08:29:21.454Z INFO webdriver: [POST] http://10.0.X.Y:4444/wd/hub/session/1b42cee1833f22e9622fbe977db8369d/url
[0-0] 2021-10-08T08:29:21.454Z INFO webdriver: DATA {
[0-0]   url: 'https://todobackend.com/client/index.html?https://todo-backend-spring4-java8.herokuapp.com/todos/'
[0-0] }
[0-0] Error in "devtools.should allow use mock feature"
Error: Expect mock to be called 1 time

Expected: 1
Received: 0
    at Context.<anonymous> (/Users/dmitry.lomakin/Sources/wdio-selenoid-examples/test/specs/example.e2e.ts:26:28)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
[0-0] 2021-10-08T08:29:27.562Z INFO webdriver: COMMAND deleteSession()
[0-0] 2021-10-08T08:29:27.562Z INFO webdriver: [DELETE] http://10.0.X.Y:4444/wd/hub/session/1b42cee1833f22e9622fbe977db8369d
[0-0] FAILED in chrome - /test/specs/example.e2e.ts
2021-10-08T08:29:27.806Z INFO @wdio/cli:launcher: Run onComplete hook

 "spec" Reporter:
------------------------------------------------------------------
[chrome 92.0.4515.131 linux #0-0] Running: chrome (v92.0.4515.131) on linux
[chrome 92.0.4515.131 linux #0-0] Session ID: 1b42cee1833f22e9622fbe977db8369d
[chrome 92.0.4515.131 linux #0-0]
[chrome 92.0.4515.131 linux #0-0] » /test/specs/example.e2e.ts
[chrome 92.0.4515.131 linux #0-0] devtools
[chrome 92.0.4515.131 linux #0-0]    ✓ should allow use puppeteer instance
[chrome 92.0.4515.131 linux #0-0]    ✖ should allow use mock feature
[chrome 92.0.4515.131 linux #0-0]
[chrome 92.0.4515.131 linux #0-0] 1 passing (8.1s)
[chrome 92.0.4515.131 linux #0-0] 1 failing
[chrome 92.0.4515.131 linux #0-0]
[chrome 92.0.4515.131 linux #0-0] 1) devtools should allow use mock feature
[chrome 92.0.4515.131 linux #0-0] Expect mock to be called 1 time

Expected: 1
Received: 0
[chrome 92.0.4515.131 linux #0-0] Error: Expect mock to be called 1 time
[chrome 92.0.4515.131 linux #0-0]
[chrome 92.0.4515.131 linux #0-0] Expected: 1
[chrome 92.0.4515.131 linux #0-0] Received: 0
[chrome 92.0.4515.131 linux #0-0]     at Context.<anonymous> (/Users/dmitry.lomakin/Sources/wdio-selenoid-examples/test/specs/example.e2e.ts:26:28)
[chrome 92.0.4515.131 linux #0-0]     at processTicksAndRejections (internal/process/task_queues.js:95:5)

Spec Files:  0 passed, 1 failed, 1 total (100% completed) in 00:00:12

2021-10-08T08:29:27.807Z INFO @wdio/local-runner: Shutting down spawned worker
2021-10-08T08:29:28.058Z INFO @wdio/local-runner: Waiting for 0 to shut down gracefully
2021-10-08T08:29:28.059Z INFO @wdio/local-runner: shutting down
npm ERR! Test failed.  See above for more details.
dmitry-lomakin commented 3 years ago

@BorisOsipov, would you please reconsider the problem before closing this issue?

BorisOsipov commented 3 years ago

The issue appears here https://github.com/webdriverio/webdriverio/blob/main/packages/wdio-devtools-service/src/utils.ts#L94

cUrl value is

[0-0] URL {
[0-0]   href: 'ws://10.10.10.1:4444/devtools/69582fa0cf807667e455ad767eb9a012',
[0-0]   origin: 'ws://10.10.10.1:4444',
[0-0]   protocol: 'ws:',
[0-0]   username: '',
[0-0]   password: '',
[0-0]   host: 'localhost:4444',
[0-0]   hostname: 'localhost',
[0-0]   port: '4444',
[0-0]   pathname: '/devtools/69582fa0cf807667e455ad767eb9a012',
[0-0]   search: '',
[0-0]   searchParams: URLSearchParams {},
[0-0]   hash: ''
[0-0] }

So it will create ChromeProtocol instance with missing websocket path part /devtools/69582fa0cf807667e455ad767eb9a012 and lead to broken 'list' command execution

christian-bromann commented 3 years ago

@dmitry-lomakin can you go in here:

/Users/dmitry.lomakin/Sources/wdio-selenoid-examples/node_modules/lighthouse/lighthouse-core/gather/connections/cri.js:97:28

and log the value that Lighthouse is trying to parse?

dmitry-lomakin commented 3 years ago

Hi @christian-bromann,

sure, the response is in the description. Here you go:

e2e-wd % curl -i http://10.0.X.Y:4444/json/list
HTTP/1.1 200 OK
Date: Fri, 08 Oct 2021 12:40:47 GMT
Content-Length: 30
Content-Type: text/plain; charset=utf-8

You are using Selenoid 1.10.5!
christian-bromann commented 3 years ago

This seems to be a Selenoid bug, @BorisOsipov can you confirm?

BorisOsipov commented 3 years ago

@christian-bromann nope. it is our bug. see https://github.com/webdriverio/webdriverio/issues/7530#issuecomment-938469999

We try to connect to ws://10.10.10.1:4444/json/list when the correct url is ws://10.10.10.1:4444/devtools/69582fa0cf807667e455ad767eb9a012/json/list

christian-bromann commented 3 years ago

http://10.0.X.Y:4444/json/list should not return a string. In fact it is a valid CDP endpoint that returns all browser contexts. The url ws://10.10.10.1:4444/devtools/69582fa0cf807667e455ad767eb9a012/json/list assumes we want to connect only to a specific one, which is not what we need.

christian-bromann commented 3 years ago

So my assumption is that Selenoid should support this endpoint and show all browser contexts it has available from all remote ends.

BorisOsipov commented 3 years ago

yes it is. see https://github.com/aerokube/selenoid/issues/1023

christian-bromann commented 3 years ago

Ok, I guess we need to follow the same suggestion as mentioned in the issue. For that I am afraid we need to copy over the ChromeProtocol class and modify it so that it mimics the same behavior but fetches the modified endpoint instead.

Any contributions that resolves the bug are highly appreciated. Please take a look into our contribution guidelines and let us know if you have any questions. Cheers!

dylanlive commented 2 years ago

Selenium 4.0 Grid also has a variant of this error: Error: Protocol JSON API error (list), status: 404

Caught up on the thread. I see the connection being made to 0.0.0.0:4444/json/list which for Selenium 4 Grid returns:

[0-0]   "value": {
[0-0]     "error": "unknown command",
[0-0]     "message": "Unable to find handler for (GET) \u002fjson\u002flist",
[0-0]     "stacktrace": ""
[0-0]   }
[0-0] }
christian-bromann commented 2 years ago

fixed in #7618, let's continue conversation there.

enliltom004 commented 2 years ago

This is still happening for Edge browsers . Can someone take a look ? https://github.com/aerokube/selenoid/issues/1023

BorisOsipov commented 2 years ago

@enliltom004 I assume you about https://github.com/aerokube/selenoid/issues/1261. https://github.com/aerokube/selenoid/issues/1023 is closed long time ago.

About 1261 - please follow @vania-pooh's advices. You have to set the path properly

vania-pooh commented 2 years ago

@enliltom004 currently we don't pack devtools daemon to Edge images. So it is not expected to work. You can create a separate issue in images repository if you need it.