Remote Firefox/IE grid nodes do not work after `.url()`

[ClaytonAstrom]

Describe the bug

I'm seeing issues when running tests on the selenium grid. On a remote grid with nodes connected, Chrome seems to work fine - however, Firefox and IE do not seem to find the body selector after navigating with the .url('http://www.google.com') command. On the nodes, the browser is open and has navigated successfully to the Google homepage.

When I run this local with selenium grid and start_server and start_process both true, all 3 browsers work just fine.

When I run this with the 3.141.59 selenium docker images, both chrome and firefox experience this issue. I can connect with VNC to see them sitting at the main page of Google, just like the hosted selenium server.

Sample test

test.js

```js import { NightwatchBrowser } from 'nightwatch'; module.exports = { 'NW test on Google': (client: NightwatchBrowser) => { client .url('http://www.google.com') .waitForElementVisible('body', 10000) .assert.title('Google') .assert.visible('input[type=text]') .setValue('input[type=text]', 'Nightwatch JS') .waitForElementVisible('input[name=btnK]', 10000) .click('input[name=btnK]') .pause(10000) .assert.containsText( '#rso div div div h3', 'Nightwatch.js | Node.js powered End-to-End testing framework' ) .end(); }, }; ```

Run with command

$ nightwatch tests/test.js --env <chrome/firefox/ie>

Verbose output

debug.log

```sh $ nightwatch --env chrome --test ./dist/tests/test.js --verbose [Test] Test Suite ================= ⠋ Connecting to localhost on port 4444... Request POST /wd/hub/session { desiredCapabilities: { browserName: 'chrome', platform: 'ANY', chromeOptions: { w3c: false }, ⠧ Connecting to localhost on port 4444... Response 200 POST /wd/hub/session (596ms) { value: { sessionId: '1441d479b9dfff7d8b4387a1d8cc6888', capabilities: { acceptInsecureCerts: false, browserName: 'chrome', browserVersion: '83.0.4103.61', chrome: { chromedriverVersion: '83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103@{#416})', userDataDir: '/tmp/.com.google.Chrome.r2iqzu' }, 'goog:chromeOptions': { debuggerAddress: 'localhost:42811' }, networkConnectionEnabled: false, pageLoadStrategy: 'normal', platformName: 'linux', proxy: {}, setWindowRect: true, strictFileInteractability: false, timeouts: { implicit: 0, pageLoad: 300000, script: 30000 }, unhandledPromptBehavior: 'dismiss and notify', 'webauthn:virtualAuthenticators': true, ℹ Connected to localhost on port 4444 (630ms). Using: chrome (83.0.4103.61) on linux platform. Received session with ID: 1441d479b9dfff7d8b4387a1d8cc6888 → Running [before]: → Completed [before]. Running: NW test on Google → Running [beforeEach]: → Completed [beforeEach]. → Running command: url ('http://www.google.com') Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/url { url: 'http://www.google.com' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/url (2729ms) { value: null } → Completed command: url ('http://www.google.com') (2730ms) → Running command: waitForElementVisible ('body', 10000) Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (32ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (23ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (26ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (26ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (30ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (28ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (25ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (21ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (19ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (24ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (22ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (23ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (23ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (23ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (23ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (20ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (25ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (20ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (19ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } Request POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements { using: 'css selector', value: 'body' } Response 200 POST /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888/elements (22ms) { value: [ { 'element-6066-11e4-a52e-4f735466cecf': '39a8da3a-bff8-440b-8e2f-3b2e8c3f9e53' } ] } ✖ Timed out while waiting for element to be present for 10000 milliseconds. - expected "visible" but got: "not found" (10047ms) at Object.NW test on Google (/Users/castrom/Documents/typescript-nightwatch-boilerplate/dist/tests/test.js:7:14) → Completed command: waitForElementVisible ('body', 10000) (10051ms) → Running [afterEach]: → Completed [afterEach]. FAILED: 1 assertions failed (12.785s) → Running [after]: → Completed [after]. → Running command: end () → Running command: session ('delete', [Function]) Request DELETE /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888 Response 200 DELETE /wd/hub/session/1441d479b9dfff7d8b4387a1d8cc6888 (68ms) { value: null } → Completed command: end () (72ms) → Completed command: session ('delete', [Function]) (69ms) _________________________________________________ TEST FAILURE: 1 assertions failed, 0 passed (13.571s) ✖ test – NW test on Google (12.785s) Timed out while waiting for element to be present for 10000 milliseconds. - expected "visible" but got: "not found" (10047ms) ```

Configuration

nightwatch.conf.js

```js const settings = { selenium: { cli_args: { 'webdriver.chrome.driver': './node_modules/.bin/chromedriver', }, host: 'localhost', port: 4444, server_path: './node_modules/selenium-server-standalone-jar/jar/selenium-server-standalone-3.141.59.jar', start_process: false, start_session: false, }, src_folders: ['./dist/tests/'], test_settings: { chrome: { desiredCapabilities: { browserName: 'chrome', chromeOptions: { w3c: false, }, }, }, firefox: { desiredCapabilities: { browserName: 'firefox', }, screenshots: { enabled: false, }, }, ie: { desiredCapabilities: { browserName: 'internet explorer', }, screenshots: { enabled: false, }, }, }, }; module.exports = settings; ```

Your Environment

Executable	Version
nightwatch --version	1.3.6
npm --version	6.11.3
yarn --version	1.22.0
node --version	10.17.0

Browser driver	Version
chromedriver	83.0.1
geckodriver	v0.26.0
ie driver	3.150.1
selenium-server	3.141.59

OS	Version
macOS Catalina	10.15.5

[ClaytonAstrom]

As a matter of A/B information, I'm able to use the same grid with a Java/Selenium framework without issue. There's something specific to Nightwatch's interaction that's causing this failure I believe

[zwbetz-gh]

The first thing I notice: the selenium key and its children should be nested within the test_settings.default, or at least under test_settings

[ClaytonAstrom]

The first thing I notice: the selenium key and its children should be nested within the test_settings.default, or at least under test_settings

Updated to test that out, doesn't seem to make a difference on the execution though, I'm still getting the same error.

nightwatch.conf.js

```js const settings = { src_folders: ['./dist/tests/'], test_settings: { chrome: { desiredCapabilities: { browserName: 'chrome', chromeOptions: { w3c: false, }, }, }, default: { selenium: { cli_args: { 'webdriver.chrome.driver': './node_modules/.bin/chromedriver', }, host: 'localhost', port: 4444, server_path: './node_modules/selenium-server-standalone-jar/jar/selenium-server-standalone-3.141.59.jar', start_process: false, start_session: false, }, }, firefox: { desiredCapabilities: { browserName: 'firefox', }, screenshots: { enabled: false, }, }, ie: { desiredCapabilities: { browserName: 'internet explorer', }, screenshots: { enabled: false, }, }, }, }; module.exports = settings; ```

[zwbetz-gh]

From the config docs

test_settings: An object in which all the test environments are defined, each overwriting test settings as needed. A default environment is always required, from which the other environments inherit settings from.

May be worth trying to nest your chrome/firefox/ie keys under default as well

[ClaytonAstrom]

From the config docs

test_settings: An object in which all the test environments are defined, each overwriting test settings as needed. A default environment is always required, from which the other environments inherit settings from.

May be worth trying to nest your chrome/firefox/ie keys under default as well

Hm, am I reading the documentation wrong? I don't think you can nest those keys? I was under the impression that all keys in the test_settings root object were the available env values?

Error: An error occurred while trying to start the Nightwatch Runner: Invalid testing environment specified: chrome. Available environments are: default

[zwbetz-gh]

Sorry, I should have been more clear. I mean like this:

Edit: see below comment

[ClaytonAstrom]

Sorry, I should have been more clear. I mean like this:

const settings = {
  src_folders: ["./dist/tests/"],
  test_settings: {
    default: {
      selenium: {
        cli_args: {
          "webdriver.chrome.driver": "./node_modules/.bin/chromedriver",
        },
        host: "localhost",
        port: 4444,
        server_path:
          "./node_modules/selenium-server-standalone-jar/jar/selenium-server-standalone-3.141.59.jar",
        start_process: false,
        start_session: false,
      },
      chrome: {
        desiredCapabilities: {
          browserName: "chrome",
          chromeOptions: {
            w3c: false,
          },
        },
      },
      firefox: {
        desiredCapabilities: {
          browserName: "firefox",
        },
        screenshots: {
          enabled: false,
        },
      },
      ie: {
        desiredCapabilities: {
          browserName: "internet explorer",
        },
        screenshots: {
          enabled: false,
        },
      },
    },
  },
};

module.exports = settings;

That's still the same issue I think, there's no other env other than default. I don't believe you're able to nest environments like that. Copying and pasting your config results in the same error anyway

[zwbetz-gh]

You are right, my apologizes. Give this a try next please.

nightwatch.conf.js

```js const settings = { src_folders: ["./dist/tests/"], test_settings: { default: { selenium: { cli_args: { "webdriver.chrome.driver": "./node_modules/.bin/chromedriver", }, host: "localhost", port: 4444, server_path: "./node_modules/selenium-server-standalone-jar/jar/selenium-server-standalone-3.141.59.jar", start_process: false, start_session: false, }, }, chrome: { desiredCapabilities: { browserName: "chrome", chromeOptions: { w3c: false, }, }, }, firefox: { desiredCapabilities: { browserName: "firefox", }, screenshots: { enabled: false, }, }, ie: { desiredCapabilities: { browserName: "internet explorer", }, screenshots: { enabled: false, }, }, }, }; module.exports = settings; ```

[beatfactor]

It's an issue with selecting the right transport. For some reason it's using Jsonwire instead of W3C Webdriver. You can try setting selenium: false as a workaround in your config, inside the firefox environment. If you feel like debugging, you can look at the code here. This is where the transport is selected.

[ClaytonAstrom]

It's an issue with selecting the right transport. For some reason it's using Jsonwire instead of W3C Webdriver. You can try setting selenium: false as a workaround in your config, inside the firefox environment. If you feel like debugging, you can look at the code here. This is where the transport is selected.

Perfect, thanks, I'll take a look into this. One question though - wouldn't disabling selenium though remove the possibility of running tests on an external grid?

[ClaytonAstrom]

@beatfactor hm, looks like SeleniumProtocol is still be set correctly at the time that the client is finalized. Still digging more into it, just posting results for posterity

[ClaytonAstrom]

Just as an update - I believe this is working ok on Selenium 4. I'm guessing something got updated for that which isn't backwards compatible with 3.x?

[ClaytonAstrom]

So hitting the 3.141.59 and 4.0.0 grids with just curl shows that the response is coming back as just { value: [<element>] } object in 3.141.59.

Still, I guess the question is should Nightwatch still be compatible with the current Selenium stable release?

[akbarchandani]

I am still facing this issue. Did anyone found any solution?

[stale[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.