karma-runner / karma-sauce-launcher

A Karma plugin. Launch any browser on SauceLabs!
MIT License
199 stars 103 forks source link

Test fails under Firefox GeckoDriver 0.31.0 #275

Open olivier-martin-sf opened 1 year ago

olivier-martin-sf commented 1 year ago

Since Sauce Labs updated (10/04/22) the Firefox GeckoDriver to version 0.31.0 our tests are failing in CI in several projects that runs some integration tests against Firefox latest.

After some investigation, it turns out that the deprecated capability highlighted in the article, --remote-debugging-port, is being sent by the webdriverio package as you can see here which prevent the driver of being instantiated and hence the session of being created as the server refuses to initiate the session when receiving an invalid capability.

CI logs

06 10 2022 19:14:21.351:ERROR [SaucelabsLauncher]: Could not quit the Saucelabs selenium connection. Failure message:
06 10 2022 19:14:21.351:ERROR [SaucelabsLauncher]: TypeError: Cannot read property 'deleteSession' of null
    at SaucelabsLauncher.<anonymous> (/home/runner/work/***-js/***-js/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:149:26)
    at Generator.next (<anonymous>)
    at /home/runner/work/***-js/***-js/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/runner/work/***-js/***-js/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:4:12)
    at SaucelabsLauncher.<anonymous> (/home/runner/work/***-js/***-js/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:141:31)
    at SaucelabsLauncher.emit (events.js:400:28)
    at SaucelabsLauncher.emit (domain.js:475:12)
    at SaucelabsLauncher.emitAsync (/home/runner/work/***-js/***-js/node_modules/karma/lib/events.js:52:10)
    at SaucelabsLauncher.BaseLauncher.kill (/home/runner/work/***-js/***-js/node_modules/karma/lib/launchers/base.js

SauceLabs test logs

Request playload to the /session endpoint:

{
    "desiredCapabilities": {
    "firefox_profile": "SOME_LONG_ID",
    "browserName": "firefox",
    "moz:firefoxOptions": {
        "args": [
        "-remote-debugging-port",
        "33759"
        ]
    }
}

The server returns an HTTP 500 status code with the following cause:

message": "Argument --remote-debugging-port can't be set via capabilities

Proposal solution

To fix this issue, we should update the webdriverio dependency so that the deprecated capability isn't sent at all.

This has been fixed in https://github.com/webdriverio/webdriverio/pull/8211 but I believe this has been shipped only since webdriverio@^7.19.5.

FYI @christian-bromann

marcoroth commented 1 year ago

Here's the workaround I used:

Since this was fixed with webdriverio 7.19.5 you can set a yarn-resolution for the webdriverio package in package.json:

  "resolutions": {
    "webdriverio": "^7.19.5"
  }

Ensure with yarn why webdriverio you are actually running the right version:

=> Found "webdriverio@7.26.0"
info Reasons this module exists
   - "karma-sauce-launcher" depends on it

And then you can use the moz:debuggerAddress capability as expected in your config:

sl_firefox_latest: {
  base: "SauceLabs",
  browserName: "firefox",
  browserVersion: "latest",
  "moz:debuggerAddress": true
},

This way you don't need to set the geckodriverVersion capability for the older driver version:

"sauce:options": {
  geckodriverVersion: "0.30.0"
}
FelixHenninger commented 1 year ago

For future reference, it appears that the "moz:debuggerAddress" setting is no longer necessary -- as of webdriverio version v7.31.1 (and possibly earlier) the package resolution mentioned above by @marcoroth is sufficient (at least it works for me™️).

Thanks to everyone for the super-useful info, this thread has been gold!

rahuljain-dev commented 8 months ago

We tried the workaround by using webdriverio different versions(latest to 7.19.5) in resolutions and direct dependency also but still facing the same issue. Can anyone help us by providing any suggestion or workaround. Thanks in advance.

dompuiu commented 8 months ago

For future reference, I got around this issue by adding the following to the package.json file:

"overrides": {
    "karma-sauce-launcher": {
      "webdriverio": "^8.29.7"
    }
  }