nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.79k stars 1.31k forks source link

Nightwatch 1.7.x with selenoid: Chrome works, firefox times out waiting for element #3004

Closed RezhaBlue closed 1 year ago

RezhaBlue commented 2 years ago

The bug

Hi all, my team and I are having issues running Firefox. The browser launches but the driver then refuses to acknowledge any elements. I've confirmed the selector (both css and xpath variant) is correct and provided more than ample time for page load, dom ready, etc. with multiple polling. No matter what Firefox times out searching for an element that is obviously there. It feels like its not looking at the webpage after browser launch but that is probably not true as commands like getTitle() are still successful. There are no iframes on the page.

This is the closest relevant nightwatch issue I could find relevant to our situation but couldnt find a suitable resolution. nightwatchjs/nightwatch#2017

Sample test

sampleTest.js

```js // Please add the sample test here module.exports = { '@tags': ['smoke_generic', 'smoke'], after(browser) { browser.end(); }, 'Basic google search start'(browser) { let searchInputLocator = 'input[title="Search"]'; const googleIt = () => { browser.url("https://google.com"); this.waitForElementVisible(searchInputLocator, function(result) { console.log(result); }).setValues(searchInputLocator,"some sample text"); }; googleIt(); } }; ```

Run with command

$ nightwatch "--verbose" "--retries" "1" "--env" "firefox" "--tag" "smoke_generic"

Verbose output

Please note these are from a real test, not the sample provided. However, at least running on my side the sample test results in the same behavior where it works in Chrome but has issues in firefox.

error.log

```txt NoSuchElementError: An error occurred while running .findElement() command on : {"value":[{"element-6066-11e4-a52e-4f735466cecf":"f28c5fcf-aaec-4682-96c6-b9eb71d50e09"}]} at logIn (/app/custom_commands/web/login_staging.js:47:12) at Object.module.exports.command (/app/custom_commands/web/login_staging.js:54:3) at processImmediate (node:internal/timers:464:21) { status: -1, value: { error: 'An error occurred while running .findElement() command on : ', message: 'An error occurred while running .findElement() command on : ', stack: 'Error\n' + ' at logIn (/app/custom_commands/web/login_staging.js:47:12)\n' + ' at Object.module.exports.command (/app/custom_commands/web/login_staging.js:54:3)\n' + ' at processImmediate (node:internal/timers:464:21)' } } NoSuchElementError: An error occurred while running .click() command on : {"value":[{"element-6066-11e4-a52e-4f735466cecf":"f28c5fcf-aaec-4682-96c6-b9eb71d50e09"}]} at logIn (/app/custom_commands/web/login_staging.js:49:10) at Object.module.exports.command (/app/custom_commands/web/login_staging.js:54:3) at processImmediate (node:internal/timers:464:21) ```

verbose.log

```txt > nightwatch "--verbose" "--retries" "1" "--env" "firefox" "--tag" "smoke_generic" [Smoke/Smoke 1a Main Menu Buttons Test] Test Suite ================================================== - Connecting to selenoid on port 4444... Request POST http://selenoid:4444 /wd/hub/session { desiredCapabilities: { browserName: 'firefox', browserVersion: '96', platformName: 'ANY', acceptInsecureCerts: true, javascriptEnabled: true, acceptSslCerts: true, marionette: true, 'moz:firefoxOptions': { prefs: { 'general.useragent.override': 'test:automation-firefox' }, log: { level: 'trace' } }, 'selenoid:options': { enableVNC: true, enableLog: true, sessionTimeout: '180s' }, name: 'Smoke/Smoke 1a Main Menu Buttons Test' } } Response 200 POST http://selenoid:4444/wd/hub/session (5113ms) { value: { sessionId: '056b46fa-1799-441a-aaf3-7274d166dfde', capabilities: { acceptInsecureCerts: true, acceptSslCerts: true, browserName: 'firefox', browserVersion: '96', javascriptEnabled: true, marionette: true, 'moz:accessibilityChecks': false, 'moz:buildID': '20220106144528', 'moz:geckodriverVersion': '0.29.1', 'moz:headless': false, 'moz:processID': 83, 'moz:profile': '/tmp/rust_mozprofileP9iA2Z', 'moz:shutdownTimeout': 60000, 'moz:useNonSpecCompliantPointerOrigin': false, 'moz:webdriverClick': true, name: 'Smoke/Smoke 1a Main Menu Buttons Test', pageLoadStrategy: 'normal', platformName: 'ANY', platformVersion: '5.10.76-linuxkit', proxy: {}, setWindowRect: true, strictFileInteractability: false, timeouts: { implicit: 0, pageLoad: 300000, script: 30000 }, unhandledPromptBehavior: 'dismiss and notify' } } } ℹ Connected to selenoid on port 4444 (5567ms). Using: firefox (96) on ANY 5.10.76-linuxkit platform. Received session with ID: 056b46fa-1799-441a-aaf3-7274d166dfde → Running [before]: → Running command: login_staging ('redacted_username', 'redacted_password') → Running command: url ('https://subdomain.domain.com') Request POST http://selenoid:4444 /wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/url { url: 'https://subdomain.domain.com' } Response 200 POST http://selenoid:4444/wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/url (4850ms) { value: null } → Completed command: url ('https://subdomain.domain..com') (4858ms) → Running command: findElement ('a[data-target="#loginModal"]', [Function]) Request POST http://selenoid:4444 /wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements { using: 'css selector', value: 'a[data-target="#loginModal"]' } Response 200 POST http://selenoid:4444/wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements (248ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '48890b52-756b-49c9-94f8-d948ae4ed056' } ] } ... repeats for the set poll frequency ... NoSuchElementError: An error occurred while running .findElement() command on : {"value":[{"element-6066-11e4-a52e-4f735466cecf":"48890b52-756b-49c9-94f8-d948ae4ed056"}]} at logIn (/app/custom_commands/web/login_staging.js:47:12) at Object.module.exports.command (/app/custom_commands/web/login_staging.js:54:3) at processImmediate (node:internal/timers:464:21) { status: -1, value: { error: 'An error occurred while running .findElement() command on : ', message: 'An error occurred while running .findElement() command on : ', stack: 'Error\n' + ' at logIn (/app/custom_commands/web/login_staging.js:47:12)\n' + ' at Object.module.exports.command (/app/custom_commands/web/login_staging.js:54:3)\n' + ' at processImmediate (node:internal/timers:464:21)' } } → Completed command: findElement ('a[data-target="#loginModal"]', [Function]) (5020ms) → Running command: click ('a[data-target="#loginModal"]') Request POST http://selenoid:4444 /wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements { using: 'css selector', value: 'a[data-target="#loginModal"]' } Response 200 POST http://selenoid:4444/wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements (44ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '48890b52-756b-49c9-94f8-d948ae4ed056' } ] ... repeats for the set poll frequency ... Request POST http://selenoid:4444 /wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements { using: 'css selector', value: 'a[data-target="#loginModal"]' } Response 200 POST http://selenoid:4444/wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements (18ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '48890b52-756b-49c9-94f8-d948ae4ed056' } ] } Request POST http://selenoid:4444 /wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements { using: 'css selector', value: 'a[data-target="#loginModal"]' } Response 200 POST http://selenoid:4444/wd/hub/session/056b46fa-1799-441a-aaf3-7274d166dfde/elements (13ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '48890b52-756b-49c9-94f8-d948ae4ed056' } ] } NoSuchElementError: An error occurred while running .click() command on : {"value":[{"element-6066-11e4-a52e-4f735466cecf":"48890b52-756b-49c9-94f8-d948ae4ed056"}]} at logIn (/app/custom_commands/web/login_staging.js:49:10) at Object.module.exports.command (/app/custom_commands/web/login_staging.js:54:3) at processImmediate (node:internal/timers:464:21) → Completed command: click ('a[data-target="#loginModal"]') (5277ms) ```

Configuration

nightwatch.json

### nightwatch.json ```js const os = require('os'); const path = require('path'); const type = process.env.TEST_TYPE.toLocaleLowerCase().trim(); const result = { live_output: true, test_workers: { enabled: process.env.TEST_WORKER_ENABLED == 'true', workers: +process.env.TEST_WORKER_COUNT != NaN ? +process.env.TEST_WORKER_COUNT : 2 }, selenium: { start_process: false, host: process.env.SELENIUM_LOCAL_REMOTE_HOST, port: process.env.SELENIUM_LOCAL_REMOTE_PORT }, test_settings: { skip_testcases_on_fail: false, end_session_on_fail: false, default: { screenshots: { enabled: true, on_failure: true, on_error: false, path: path.join('output', type, process.env.CURRENT_TIMESTAMP, 'screenshots') } }, chrome: { desiredCapabilities: { browserName: 'chrome', acceptInsecureCerts: true, javascriptEnabled: true, acceptSslCerts: true, browserVersion: '97', platformName: 'ANY', 'selenoid:options': { enableVNC: true, // enableVideo: true, sessionTimeout: '180s' }, chromeOptions: { w3c: false, args: [ 'no-sandbox', 'disable-setuid-sandbox', 'start-maximized', 'disable-gpu', '--user-agent=test:automation-chrome' ] } } }, firefox: { desiredCapabilities: { browserName: 'firefox', browserVersion: '96', platformName: 'ANY', acceptInsecureCerts: true, javascriptEnabled: true, acceptSslCerts: true, marionette: true, 'moz:firefoxOptions': { prefs: { 'general.useragent.override': 'test:automation-firefox' }, log: { level: 'trace' } }, 'selenoid:options': { enableVNC: true, //enableVideo: true, enableLog: true, sessionTimeout: '180s' } } }, edge: { desiredCapabilities: { browserName: 'MicrosoftEdge', browserVersion: '97', platformName: 'ANY', acceptInsecureCerts: true, javascriptEnabled: true, acceptSslCerts: true, 'selenoid:options': { enableVNC: true, //enableVideo: true, enableLog: true, sessionTimeout: '180s' }, edgeOptions: { w3c: false, args: [ 'no-sandbox', 'disable-setuid-sandbox', 'start-maximized', 'disable-gpu', '--user-agent=test:automation-edge' ] } } } } }; if (os.type() == 'Darwin') { result.test_settings.safari = { desiredCapabilities: { browserName: 'safari', browserVersion: '15', platformName: 'ANY', acceptInsecureCerts: true, javascriptEnabled: true, acceptSslCerts: true, 'selenoid:options': { enableVNC: true, sessionTimeout: '180s' }, safariOptions: { w3c: false, args: [ 'no-sandbox', 'disable-setuid-sandbox', 'start-maximized', 'disable-gpu', '--user-agent=test:automation-safari' ] } } }; } module.exports = result; ```

browsers.json

### Browser configs for Selenoid ```js { "firefox": { "default": "96.0", "versions": { "96.0": { "image": "selenoid/vnc_firefox:96.0", "port": "4444", "path": "/wd/hub", "tmpfs": {"/tmp":"size=512m"} } } }, "chrome": { "default": "97.0", "versions": { "97.0": { "image": "selenoid/vnc_chrome:97.0", "port": "4444", "tmpfs": {"/tmp":"size=512m"} } } }, "MicrosoftEdge": { "default": "97.0", "versions": { "97.0": { "image": "browsers/edge:97.0", "port": "4444", "tmpfs": {"/tmp":"size=512m"} } } }, "safari": { "default": "15.0", "versions": { "15.0": { "image": "browsers/safari:15.0", "port": "4444", "tmpfs": {"/tmp":"size=512m"} } } } } ```

Environment

We've tried on different people's computers (mac m1, mac monterey, ubuntu 20, windows 10, docker:node latest) and both dockerized/containerized (node:lts-slim image) and without, and in our local, testing, staging, and prod environments. None of these factors seem to alter the behavior.

the browser images used include selenoid/firefox:87.0 through to the latest selenoid/firefox:96.0 docker images, and selenoid/chrome 90+

Executable Version
nightwatch --version 1.7.10+
npm --version 8.1.2
yarn --version 1.22.15
node --version 16.13.2
Browser driver Version
NAME VERSION
chromedriver 90+
geckodriver v0.29.1
firefox 87+
OS Version
NAME VERSION
macOS monterey (M1/arm64) 12.1
macOS monterey 12.1
Ubuntu 20
windows 10 1607+
node:lts-slim 16.13.2
RezhaBlue commented 2 years ago

We have moved to an implementation using selenium-hub and are seeing the same issue spoke of in #2017 Is there anyone who can speak to progress or workarounds for this? This seems to effect all Nightwatch API calls that use the getElementProperty method or property endpoint

AutomatedTester commented 1 year ago

we have updated how we work with mobile using a mobile helper and with appium in Nightwatch 2. COuld you please retry with the latest and see if this is still an issue.

AutomatedTester commented 1 year ago

Closing as info not provided