.element() return value inconsistent between Chrome or Firefox

[mwilc0x]

Hi,

This is similar to #2017.

Am seeing inconsistent output from call to element depending on if running with Chrome or Firefox. More details below on output, configuration, and test code.

I am a bit stumped on what changes I may need to make, or if this is something else. Any help, suggestions or advice would be greatly appreciated. Thank you!

node v14.18.1
nightwatch ^1.7.11
chrome (95.0.4638.69) on Linux platform.
firefox (78.15.0) on linux 5.10.46-5rodete1-amd64

Test Code

this.element('css selector', selector, (frame) => {
  if (frame.status == -1) {
    this.log(frame.error, true);
  }
  this.frame({ELEMENT: frame.value.ELEMENT}, () => {
    this.log(`Switching to ${iframeMsg}`);
    callback && callback();
  });
});

Output

Firefox

{
  value: {
    "element-6066-11e4-a52e-4f735466cecf": "6c47eead-5b9b-4297-a151-235d86e8d53c",
  },
}

Chrome

{
  sessionId: "17bcc89fa665532574182cb0bc9f848b",
  status: 0,
  value: {
    ELEMENT: "0.6987184302189291-1",
  },
}

Configuration

nightwatch.conf.js

const chromeDriver = require('chromedriver');
const geckoDriver = require('geckodriver');
const seleniumServer = require('selenium-server');

module.exports = {
  src_folders: ['test/e2e/tests'],
  globals_path: 'globals.js',
  page_objects_path: 'test/e2e/pages',

  test_settings: {
    default: {
      launch_url: 'http://localhost:8000/examples/sample-pub/1',
      custom_commands_path: 'test/e2e/commands',
      skip_testcases_on_fail: false,
    },

    chrome: {
      globals: {
        webdriverProcess: 'chromedriver',
      },

      desiredCapabilities: {
        browserName: 'chrome',
        chromeOptions: {
          args: ['--headless'],
        },
      },

      webdriver: {
        start_process: true,
        server_path: chromeDriver.path,
        port: 9515,
      },
    },

    firefox: {
      globals: {
        webdriverProcess: 'geckodriver',
      },

      desiredCapabilities: {
        browserName: 'firefox',
        acceptInsecureCerts: true,
        alwaysMatch: {
          'moz:firefoxOptions': {
            args: ['-headless'],
          },
        },
      },

      webdriver: {
        start_process: true,
        server_path: 'node_modules/.bin/geckodriver',
        cli_args: ['--log', 'debug'],
        port: 4444,
      },
    },

    selenium: {
      // Selenium Server is running locally and is managed by Nightwatch
      selenium: {
        start_process: true,
        port: 4444,
        server_path: seleniumServer.path,
        cli_args: {
          'webdriver.chrome.driver': chromeDriver.path,
          'webdriver.gecko.driver': geckoDriver.path,
        },
      },
      webdriver: {
        start_process: false,
      },
    },

    'selenium.chrome': {
      extends: 'selenium',
      desiredCapabilities: {
        browserName: 'chrome',
        chromeOptions: {
          w3c: false,
          args: ['--headless'],
        },
      },
    },

    'selenium.firefox': {
      extends: 'selenium',
      desiredCapabilities: {
        browserName: 'firefox',
      },
    },
  },
};

[gravityvi]

Can you set w3c:true and try again?

[mwilc0x]

Hi @gravityvi , thanks for your response!

Where should the w3c:true setting be? Currently the only w3c setting I have is for selenium.chrome:

'selenium.chrome': {
  extends: 'selenium',
  desiredCapabilities: {
    browserName: 'chrome',
    chromeOptions: {
      w3c: false,
      args: ['--headless'],
    },
  },

Also, a quick search and found this comment which seems to address the specific problem am still seeing.

this happens because Chrome still uses JsonWire for the time being, and so you'll still get {'ELEMENT': '…'} until they switch to WebDriver.

Am wondering if there is still some discrepancy with JsonWire and WebDriver with Chrome/Firefox?

[gravityvi]

so chrome with flag w3c:false uses JsonWire protocol, while firefox does not support JsonWire anymore. By default chrome would use w3c:true.

[mwilc0x]

I see here the JsonWire responses matches what I am seeing when running with chrome currently.

I switched those to w3c:true and checked that there is not any other w3c:false but am still seeing the JsonWire response when debugging.

Is the format that is being returned for GeckoDriver (see below) the expected output that should be tested with going forward?

{
  value: {
    "element-6066-11e4-a52e-4f735466cecf": "6c47eead-5b9b-4297-a151-235d86e8d53c",
  },
}

I was also thinking of upgrading nightwatch to v2, but we are using a gulp plugin that hasn't been updated in a while and I think I would need to re-write it as some things have changed. Is it advised to upgrade to v2 asap?

I have also noticed some othe quirks between chrome and firefox, like .submitForm() not working in Firefox as it was for chrome. It seems there is some mis-configuration somewhere, or maybe some other issues.

[mwilc0x]

Ah! Ok, setting w3c:true was correct :) sorry for the confusion!

      desiredCapabilities: {
        browserName: 'chrome',
        chromeOptions: {
          args: ['--headless'],
          w3c: true
        },
      },

Is this the recommended way we should test going forward using WebDriver and not legacy JsonWire? Should we upgrade to v2?

[mwilc0x]

I am going to mark this as closed for now.

The solution to get Chrome and Firefox in sync was setting w3c:true for chrome as mentioned in https://github.com/nightwatchjs/nightwatch/issues/2934#issuecomment-956287776.

[gravityvi]

@michaelwilcox upgrading to v2 is recommended, but there might be changes required in test scripts. You can refer this breaking changes