nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.78k stars 1.31k forks source link

waitForElementVisible failed with chromedriver 75.0.3770.8 #2082

Closed vlad-vinogradov closed 5 years ago

vlad-vinogradov commented 5 years ago

Looks like Nightwatch 1.1.7 doesn't recognize the new W3C response/identifier format:

{ value:
   [ { 'element-6066-11e4-a52e-4f735466cecf': 'c125c68d-c4da-4422-abf5-04757feb6961' } ] }

testGoogle.js:

module.exports = {
  'Demo test Google' : function (browser) {
    browser
      .url('https://www.google.com')
      .waitForElementVisible('body', 4000)
      .setValue('input[type=text]', 'nightwatch')
      .waitForElementVisible('input[name=btnK]', 5000)
      .click('input[name=btnK')
      .pause(3000)
      .assert.containsText('#search', 'Nightwatch.js')
      .end();
  }
};

Here is the output from Nightwatch:

   Request POST  /wd/hub/session
   { desiredCapabilities:
      { browserName: 'chrome',
        acceptSslCerts: true,
        platform: 'ANY',
        applicationName: 'chrome-unstable',
        javascriptEnabled: true,
        name: 'Test Google' } }
   Response 200 POST /wd/hub/session (808ms)
   { value:
      { sessionId: 'f07a12f028be56fe188dd3de20e54cfe',
        capabilities:
         { acceptInsecureCerts: false,
           browserName: 'chrome',
           browserVersion: '75.0.3770.18',
           chrome:
            { chromedriverVersion:
               '75.0.3770.8 (681f24ea911fe754973dda2fdc6d2a2e159dd300-refs/branch-heads/3770@{#40})',
              userDataDir: '/tmp/.com.google.Chrome.SgGWNM' },
           'goog:chromeOptions': { debuggerAddress: 'localhost:37693' },
           networkConnectionEnabled: false,
           pageLoadStrategy: 'normal',
           platformName: 'linux',
           proxy: {},
           setWindowRect: true,
           strictFileInteractability: false,
           timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
           unhandledPromptBehavior: 'dismiss and notify',
           'webdriver.remote.sessionid': 'f07a12f028be56fe188dd3de20e54cfe' } } }
 Received session with ID: f07a12f028be56fe188dd3de20e54cfe

 -> Running [before]:
 -> Completed [before].
Running:  Demo test Google

 -> Running [beforeEach]:
 -> Completed [beforeEach].

 -> Running command: url ('https://www.google.com')
   Request POST  /wd/hub/session/f07a12f028be56fe188dd3de20e54cfe/url  
   { url: 'https://www.google.com' }
   Response 200 POST /wd/hub/session/f07a12f028be56fe188dd3de20e54cfe/url (3378ms)
   { value: null }
 -> Completed command url ('https://www.google.com') (3381ms)

 -> Running command: waitForElementVisible ('body', 4000)
   Request POST  /wd/hub/session/f07a12f028be56fe188dd3de20e54cfe/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/f07a12f028be56fe188dd3de20e54cfe/elements (35ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': 'c125c68d-c4da-4422-abf5-04757feb6961' } ] }
   Request POST  /wd/hub/session/f07a12f028be56fe188dd3de20e54cfe/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/f07a12f028be56fe188dd3de20e54cfe/elements (29ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': 'c125c68d-c4da-4422-abf5-04757feb6961' } ] }

...

And here is successful output when using chromedriver 74.0.3729.6: ...

...
 -> Running command: waitForElementVisible ('body', 1000)
   Request POST  /wd/hub/session/7f35786ac6089c26120693e2f33680ce/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/7f35786ac6089c26120693e2f33680ce/elements (39ms)
   { sessionId: '7f35786ac6089c26120693e2f33680ce',
     status: 0,
     value: [ { ELEMENT: '0.8304314778554673-1' } ] }
   Request GET  /wd/hub/session/7f35786ac6089c26120693e2f33680ce/element/0.8304314778554673-1/displayed  
   Response 200 GET /wd/hub/session/7f35786ac6089c26120693e2f33680ce/element/0.8304314778554673-1/displayed (28ms)
   { sessionId: '7f35786ac6089c26120693e2f33680ce',
     status: 0,
     value: true }
V Element <body> was visible after 70 milliseconds.
 -> Completed command waitForElementVisible ('body', 1000) (73ms)
vlad-vinogradov commented 5 years ago

... the same problem with waitForElementPresent

vlad-vinogradov commented 5 years ago

... and the same with Firefox and geckodriver v0.24.0:

   Request POST  /wd/hub/session  
   { desiredCapabilities:
      { browserName: 'firefox',
        acceptSslCerts: true,
        platform: 'ANY',
        applicationName: 'firefox-latest',
        javascriptEnabled: true,
        'moz:firefoxOptions':
         { binary: '/opt/selenium-firefox/selenium-firefox-66.0.3',
           prefs: { 'permissions.default.desktop-notification': 1 } },
        name: 'Test Google' } }
   Response 200 POST /wd/hub/session (16438ms)
   { value:
      { sessionId: 'a847ccf3-899f-4995-920e-39213f771db8',
        capabilities:
         { acceptInsecureCerts: false,
           browserName: 'firefox',
           browserVersion: '66.0.3',
           'moz:accessibilityChecks': false,
           'moz:geckodriverVersion': '0.24.0',
           'moz:headless': false,
           'moz:processID': 17189,
           'moz:profile': '/tmp/rust_mozprofile.5QT2tP70lq6K',
           'moz:shutdownTimeout': 60000,
           'moz:useNonSpecCompliantPointerOrigin': false,
           'moz:webdriverClick': true,
           pageLoadStrategy: 'normal',
           platformName: 'linux',
           platformVersion: '4.15.0-47-generic',
           rotatable: false,
           setWindowRect: true,
           strictFileInteractability: false,
           timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
           unhandledPromptBehavior: 'dismiss and notify',
           'webdriver.remote.sessionid': 'a847ccf3-899f-4995-920e-39213f771db8' } } }
 Received session with ID: a847ccf3-899f-4995-920e-39213f771db8

-> Running [before]:
-> Completed [before].
Running:  Demo test Google

-> Running [beforeEach]:
-> Completed [beforeEach].

-> Running command: url ('https://www.google.com')
   Request POST  /wd/hub/session/a847ccf3-899f-4995-920e-39213f771db8/url  
   { url: 'https://www.google.com' }
   Response 200 POST /wd/hub/session/a847ccf3-899f-4995-920e-39213f771db8/url (6314ms)
   { value: null }
-> Completed command url ('https://www.google.com') (6316ms)

-> Running command: waitForElementVisible ('body', 4000)
   Request POST  /wd/hub/session/a847ccf3-899f-4995-920e-39213f771db8/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/a847ccf3-899f-4995-920e-39213f771db8/elements (44ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': 'b1335356-fb6b-4409-a3b6-57391da6de7d' } ] }
   Request POST  /wd/hub/session/a847ccf3-899f-4995-920e-39213f771db8/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/a847ccf3-899f-4995-920e-39213f771db8/elements (20ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': 'b1335356-fb6b-4409-a3b6-57391da6de7d' } ] }

. . .
beatfactor commented 5 years ago

It does.

However, Chrome (stable) still uses JSONWire protocol so we cannot switch yet to full W3C Webdriver. But you can try to switch manually by setting:

webdriver: {
  use_legacy_jsonwire: true
}

https://github.com/nightwatchjs/nightwatch/blob/master/lib/settings/defaults.js#L109

vlad-vinogradov commented 5 years ago

I have tried to play around manually with "use_legacy_jsonwire" in the node_modules/nightwatch/lib/settings/defaults.js, but without success.

This is test configuration:

Google Chrome (unstable): 76.0.3795.3-1
ChromeDriver:  75.0.3770.8
Selenium Grid: selenium-server-standalone-3.141.59.jar
Nightwatch: 1.1.10

tests/google.js
---------------
module.exports = {
  'demo-test' : function (browser)
  {
     browser.url('https://www.google.com/')
      .waitForElementVisible('body', 6000)
      .setValue('input[type=text]', ['webdriver', browser.Keys.ENTER])
      .pause(5000)
      .assert.containsText('#search', 'Selenium')
      .perform( function() { console.log("Test - OK\n"); } )
      .end();
  }
};

nightwatch.json (nightwatch --env chrome-dev)
---------------
{
  "src_folders" :   ["tests"],
  "output_folder" : "reports",
  "custom_commands_path" :   "",
  "custom_assertions_path" : "",
  "globals_path" : "",

  "test_settings":
  {
    "default": {
      "selenium_port":  4444,
      "selenium_host":  "localhost",
      "silent":         true,
      "desiredCapabilities":
      {
        "browserName":       "chrome",
        "applicationName":   "chrome-new",
        "javascriptEnabled": true,
        "acceptSslCerts":    true
      }
    },

    "chrome-old":  { "desiredCapabilities": { "browserName": "chrome",  "applicationName": "chrome-old"  } },
    "chrome-new":  { "desiredCapabilities": { "browserName": "chrome",  "applicationName": "chrome-new"  } },
    "chrome-dev":  { "desiredCapabilities": { "browserName": "chrome",  "applicationName": "chrome-dev"  } },

    "firefox-old": { "desiredCapabilities": { "browserName": "firefox", "applicationName": "firefox-old" } },
    "firefox-new": { "desiredCapabilities": { "browserName": "firefox", "applicationName": "firefox-new" } },
    "firefox-dev": { "desiredCapabilities": { "browserName": "firefox", "applicationName": "firefox-dev" } }
  }

hubConfig
---------
{
    "host": "0.0.0.0",
    "port":  4444,
    "role": "hub",
    "maxSession": 10,
    "newSessionWaitTimeout": -1,
    "capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
    "throwOnCapabilityNotPresent": true,
    "jettyMaxThreads": -1,
    "cleanUpCycle": 5000,
    "browserTimeout": 0,
    "timeout": 1800,
    "debug": false
}

nodeConfig
----------
{
    "capabilities": [
        {
            "applicationName": "chrome-dev",
            "browserVersion": "76.0.3795.3-1",
            "browserName": "chrome",
            "platformName": "linux",
            "maxInstances": 1,
            "seleniumProtocol": "WebDriver",
            "chrome_binary": "/opt/devservices/grid/chrome/76.0.3795.3-1/google-chrome"
        }
    ],
    "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
    "maxSession": 10,
    "host": "0.0.0.0",
    "port": 20004,
    "register": true,
    "registerCycle": 5000,
    "hub": "http://localhost:4444",
    "nodePolling": 5000,
    "role": "node",
    "unregisterIfStillDownAfter": 60000,
    "downPollingLimit": 2,
    "debug": false,
    "servlets" : []
}

(change "chrome_binary" appropriately)
vlad-vinogradov commented 5 years ago

...

java -jar /opt/devservices/grid/selenium-server-standalone-3.141.59.jar \
     -host 0.0.0.0 -maxSession 10 -port 4444 \
     -role hub -hubConfig /home/vagrant/DevServicesGrid/json/hub.json

java -Dwebdriver.chrome.driver=/opt/devservices/grid/chrome/76.0.3795.3-1/chromedriver \
    -jar /opt/devservices/grid/selenium-server-standalone-3.141.59.jar \
    -host 0.0.0.0 -maxSession 10 -hubPort 4444 \
    -role node -nodeConfig /home/vagrant/DevServicesGrid/json/node-chrome-76.0.3795.3-1.json
beatfactor commented 5 years ago

You need to define the above setting in your nightwatch.json (or .conf.js) file.

vlad-vinogradov commented 5 years ago

This option has no effect on the chromedriver when using Selenium Grid:

nightwatch.json
---------------
{
  "src_folders" :   ["tests"],
  "output_folder" : "reports",
  "custom_commands_path" :   "",
  "custom_assertions_path" : "",
  "globals_path" : "",

  "test_settings":
  {
    "default": {

      "webdriver": { "use_legacy_jsonwire": true },

      "selenium_port":  4444,
      "selenium_host":  "localhost",
      "silent":         true,
      "desiredCapabilities":
      {
        "browserName":       "chrome",
        "applicationName":   "chrome-new",
        "javascriptEnabled": true,
        "acceptSslCerts":    true
      }
    },

    "chrome-old":  { "desiredCapabilities": { "browserName": "chrome",  "applicationName": "chrome-old"  } },
    "chrome-new":  { "desiredCapabilities": { "browserName": "chrome",  "applicationName": "chrome-new"  } },
    "chrome-dev":  {
       "webdriver": { "use_legacy_jsonwire": true },
       "desiredCapabilities": { "browserName": "chrome",  "applicationName": "chrome-dev"  }
    },
    "firefox-old": { "desiredCapabilities": { "browserName": "firefox", "applicationName": "firefox-old" } },
    "firefox-new": { "desiredCapabilities": { "browserName": "firefox", "applicationName": "firefox-new" } },
    "firefox-dev": { "desiredCapabilities": { "browserName": "firefox", "applicationName": "firefox-dev" } }
  }
}

OUTPUT
------

   Request POST  /wd/hub/session  
   { desiredCapabilities:
      { browserName: 'chrome',
        acceptSslCerts: true,
        platform: 'ANY',
        applicationName: 'chrome-dev',
        javascriptEnabled: true,
        name: 'Google' } }
   Response 200 POST /wd/hub/session (808ms)
   { value:
      { sessionId: '758117a5e479dc9c2bd354922b00049b',
        capabilities:
         { acceptInsecureCerts: false,
           browserName: 'chrome',
           browserVersion: '76.0.3795.3',
           chrome:
            { chromedriverVersion:
               '75.0.3770.8 (681f24ea911fe754973dda2fdc6d2a2e159dd300-refs/branch-heads/3770@{#40})',
              userDataDir: '/tmp/.com.google.Chrome.qLYrEk' },
           'goog:chromeOptions': { debuggerAddress: 'localhost:33055' },
           networkConnectionEnabled: false,
           pageLoadStrategy: 'normal',
           platformName: 'linux',
           proxy: {},
           setWindowRect: true,
           strictFileInteractability: false,
           timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
           unhandledPromptBehavior: 'dismiss and notify',
           'webdriver.remote.sessionid': '758117a5e479dc9c2bd354922b00049b' } } }
 Received session with ID: 758117a5e479dc9c2bd354922b00049b

 -> Running [before]:
 -> Completed [before].
Running:  demo-test

 -> Running [beforeEach]:
 -> Completed [beforeEach].

 -> Running command: url ('https://www.google.com/')
   Request POST  /wd/hub/session/758117a5e479dc9c2bd354922b00049b/url  
   { url: 'https://www.google.com/' }
   Response 200 POST /wd/hub/session/758117a5e479dc9c2bd354922b00049b/url (3818ms)
   { value: null }
 -> Completed command url ('https://www.google.com/') (3820ms)

 -> Running command: waitForElementVisible ('body', 6000)
   Request POST  /wd/hub/session/758117a5e479dc9c2bd354922b00049b/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/758117a5e479dc9c2bd354922b00049b/elements (36ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': '07bbf9d0-b398-4db3-ac8e-0ccdd127c690' } ] }
   Request POST  /wd/hub/session/758117a5e479dc9c2bd354922b00049b/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/758117a5e479dc9c2bd354922b00049b/elements (28ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': '07bbf9d0-b398-4db3-ac8e-0ccdd127c690' } ] }

   . . .

   Request POST  /wd/hub/session/758117a5e479dc9c2bd354922b00049b/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/758117a5e479dc9c2bd354922b00049b/elements (26ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': '07bbf9d0-b398-4db3-ac8e-0ccdd127c690' } ] }
 -> Completed command waitForElementVisible ('body', 6000) (6384ms)
   NoSuchElementError: Error while running "waitForElementVisible" command: Timed out while waiting for element "body" with "css selector" to be present for 6000 milliseconds.
       at processTicksAndRejections (internal/process/next_tick.js:81:5)
 -> Running [afterEach]:
 -> Completed [afterEach].

FAILED: 1 errors (10.213s)
   NoSuchElementError: Error while running "waitForElementVisible" command: Timed out while waiting for element "body" with "css selector" to be present for 6000 milliseconds.
       at processTicksAndRejections (internal/process/next_tick.js:81:5)
 -> Running [after]:
 -> Completed [after].

 -> Running command: end ([Function])

 -> Running command: session ('delete', [Function])
   Request DELETE  /wd/hub/session/758117a5e479dc9c2bd354922b00049b  
   Response 200 DELETE /wd/hub/session/758117a5e479dc9c2bd354922b00049b (65ms)
   { value: null }
 -> Completed command end ([Function]) (70ms)
 -> Completed command session ('delete', [Function]) (67ms)
beatfactor commented 5 years ago

Try this:

"webdriver": { 
  "start_process": false,
  "default_path_prefix": "/wd/hub",
  "use_legacy_jsonwire": false,
  "port": 4444,
  "host": "localhost"
},
beatfactor commented 5 years ago

The code for creating the transport is here: https://github.com/nightwatchjs/nightwatch/blob/master/lib/transport/transport.js#L286. Since your use case is a bit more irregular, maybe you could have a look and suggest appropriate changes?

vlad-vinogradov commented 5 years ago

According to the W3C WebDriver Specification Selenium 3 and higher (hub or standalone) is just an "intermediate node" acting as a proxy. It can serve requests in accordance with W3C WebDriver Protocol and has some legacy features.
The same applies to the Chromedriver, Geckodriver and others. For Nightwatch they are all just WebDriver-servers.

The required protocol type depends mainly on the web-browser version. So you need some parameter with which you can explicitly specify the required protocol in the environment section. And need some procedure for calculating the default value, which can be changed in a year or two.

For example, you can add a "protocol" parameter to environment section with values such as "WebDriver", "JsonWire", etc. And you can leave the current algorithm as a procedure for calculating the default value.

vlad-vinogradov commented 5 years ago

... and I have manually forced the WebdriverProtocol in the transport.js and run the test on the latest versions of Firefox (69.0a1, geckodriver 0.24.0) and Google Chrome (76.0.3795.3-1, chromedriver 75.0.3770.8).

There are some compatibility issues with W3C Specification. API source code checking is required.

Chromedriver: waitForElementVisible('body', 6000):

 -> Running command: waitForElementVisible ('body', 6000)
   Request POST  /wd/hub/session/becfe806d69d0f763d82a092aeda35a8/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /wd/hub/session/becfe806d69d0f763d82a092aeda35a8/elements (40ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': '80854035-6b0f-4831-96ec-f07184c10907' } ] }
   Request GET  /wd/hub/session/becfe806d69d0f763d82a092aeda35a8/element/80854035-6b0f-4831-96ec-f07184c10907/displayed  
   Response 404 GET /wd/hub/session/becfe806d69d0f763d82a092aeda35a8/element/80854035-6b0f-4831-96ec-f07184c10907/displayed (20ms)
   { value:
      { error: 'unknown command',
        message:
         'unknown command: Cannot call non W3C standard command while in W3C mode',
        stacktrace: '#0 0x5565ba9ec299 <unknown>\n' } }
 Error while running .isElementDisplayed() protocol action: unknown command: Cannot call non W3C standard command while in W3C mode

Chromedriver: setValue('input[type=text]', ['webdriver', browser.Keys.ENTER]):

 -> Running command: setValue ('input[type=text]', {0, 1})
   Request POST  /wd/hub/session/b090df189904bdf57557bcc6ee84e61e/elements  
   { using: 'css selector', value: 'input[type=text]' }
   Response 200 POST /wd/hub/session/b090df189904bdf57557bcc6ee84e61e/elements (31ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': '6bfdbd05-c1d8-454a-a340-ada38cb5951d' } ] }
   Request POST  /wd/hub/session/b090df189904bdf57557bcc6ee84e61e/element/6bfdbd05-c1d8-454a-a340-ada38cb5951d/value  
   { text: [ 'webdriver', '...' ] }
   Response 400 POST /wd/hub/session/b090df189904bdf57557bcc6ee84e61e/element/6bfdbd05-c1d8-454a-a340-ada38cb5951d/value (18ms)
   { value:
      { error: 'invalid argument',
        message:
         "invalid argument: 'text' must be a string\n  (Session info: chrome=76.0.3795.3)",
        stacktrace: '#0 0x560743595299 <unknown>\n' } }
 Error while running .setElementValue() protocol action: invalid argument: 'text' must be a string
  (Session info: chrome=76.0.3795.3)

Geckodriver: setValue('input[type=text]', ['webdriver', browser.Keys.ENTER]):

 -> Running command: setValue ('input[type=text]', {0, 1})
   Request POST  /wd/hub/session/a691b7a1-5c6b-4297-a1f1-ced6bc3415e5/elements  
   { using: 'css selector', value: 'input[type=text]' }
   Response 200 POST /wd/hub/session/a691b7a1-5c6b-4297-a1f1-ced6bc3415e5/elements (17ms)
   { value:
      [ { 'element-6066-11e4-a52e-4f735466cecf': 'a774ed3e-c24c-4172-83e2-041e0837ee9d' } ] }
   Request POST  /wd/hub/session/a691b7a1-5c6b-4297-a1f1-ced6bc3415e5/element/a774ed3e-c24c-4172-83e2-041e0837ee9d/value  
   { text: [ 'webdriver', '...' ] }
   Response 400 POST /wd/hub/session/a691b7a1-5c6b-4297-a1f1-ced6bc3415e5/element/a774ed3e-c24c-4172-83e2-041e0837ee9d/value (51ms)
   { value:
      { error: 'invalid argument',
        message:
         'invalid type: sequence, expected a string at line 1 column 8',
        stacktrace: '' } }
 Error while running .setElementValue() protocol action: invalid type: sequence, expected a string at line 1 column 8

14.3 Element Send Keys:

The remote end steps for Element Send Keys are:

  • Let text be the result of getting a property called "text" from the parameters argument.
  • If text is not a String, return an error with error code invalid argument. ...
beatfactor commented 5 years ago

ChromeDriver doesn't implement element /displayed endpoint. setValue should be fixed.

vlad-vinogradov commented 5 years ago

It can be useful (concerning lib/transport/transport.js):

There are also selenium2.js and jsonwire.js...
Difficult to guess how Selenium 4 will behave.

So, use_legacy_jsonwire alone is not enough to cover all possible Selenium use cases.

vlad-vinogradov commented 5 years ago

Currently it's hard to support all possible use cases of Selenium Grid/Standalone with different web browser's versions. This requires the dynamic selection of the protocol to be used in the session establishment step.
In some cases at the moment it is easier to use your own temporary fork. So I close this issue.

penguin1214 commented 3 years ago

Looks like Chrome is using WebDriver W3C protocol by default now. Is there any plan for Nightwatch to fully switch to W3C standard? I'm still running into this issue for Firefox.