Demo test fails, google.com lacks <button[name=btnG]>

xurizaemon commented 5 years ago

Hi there! I've read instructions, searched issue queue and hopefully done what I need to make your job easier. This is probably a docs/marketing issue, and ultimately due to NightwatchJS not being in control of Google's DOM.

For me, the example test on the front page of nightwatchjs.org fails because there is no element <button[name=btnG]> on https://google.com

Expectations

I expect that the example test will pass
I see that the example test fails

Proposed resolution

Update the example test with one that works, OR if suitable replace it with a test that depends on HTML you're in control of.

Versions

nightwatch v0.9.21
nodejs v8.11.3
ubuntu 18.04
selenium 3.141.0

Test code

Obtained just now from front page of https://nightwatchjs.org

module.exports = {
  'Demo test Google' : function (client) {
    client
      .url('http://www.google.com')
      .waitForElementVisible('body', 1000)
      .assert.title('Google')
      .assert.visible('input[type=text]')
      .setValue('input[type=text]', 'rembrandt van rijn')
      .waitForElementVisible('button[name=btnG]', 1000)
      .click('button[name=btnG]')
      .pause(1000)
      .assert.containsText('ol#rso li:first-child',
        'Rembrandt - Wikipedia')
      .end();
  }
};

Output

08:56 $ ~/Projects/example/node_modules/nightwatch/bin/nightwatch --verbose tests/NightWatch/google.js 

[Night Watch / Google] Test Suite
=====================================

Running:  Demo test Google
INFO Request: POST /wd/hub/session 
 - data:  {"desiredCapabilities":{"browserName":"chrome","javascriptEnabled":true,"acceptSslCerts":true,"platform":"ANY","marionette":true,"name":"Night Watch / Google"}} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":160}
INFO Response 200 POST /wd/hub/session (1579ms) { status: 0,
  sessionId: 'c063102d2b6f37a80a399192e9005705',
  value: 
   { acceptInsecureCerts: false,
     acceptSslCerts: false,
     applicationCacheEnabled: false,
     browserConnectionEnabled: false,
     browserName: 'chrome',
     chrome: 
      { chromedriverVersion: '2.44.609551 (5d576e9a44fe4c5b6a07e568f1ebc753f1214634)',
        userDataDir: '/tmp/.org.chromium.Chromium.c8kMYE' },
     cssSelectorsEnabled: true,
     databaseEnabled: false,
     'goog:chromeOptions': { debuggerAddress: 'localhost:45183' },
     handlesAlerts: true,
     hasTouchScreen: false,
     javascriptEnabled: true,
     locationContextEnabled: true,
     mobileEmulationEnabled: false,
     nativeEvents: true,
     networkConnectionEnabled: false,
     pageLoadStrategy: 'normal',
     platform: 'Linux',
     rotatable: false,
     setWindowRect: true,
     takesHeapSnapshot: true,
     takesScreenshot: true,
     unexpectedAlertBehaviour: 'ignore',
     version: '70.0.3538.77',
     webStorageEnabled: true,
     'webdriver.remote.sessionid': 'c063102d2b6f37a80a399192e9005705' } }
INFO Got sessionId from selenium c063102d2b6f37a80a399192e9005705
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/url 
 - data:  {"url":"http://www.google.com"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":31}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/url (2422ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: null }
LOG     → Completed command url (2429 ms)
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements 
 - data:  {"using":"css selector","value":"body"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":39}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements (35ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: [ { ELEMENT: '0.5168266409339404-1' } ] }
INFO Request: GET /wd/hub/session/c063102d2b6f37a80a399192e9005705/element/0.5168266409339404-1/displayed 
 - data:   
 - headers:  {"Accept":"application/json"}
INFO Response 200 GET /wd/hub/session/c063102d2b6f37a80a399192e9005705/element/0.5168266409339404-1/displayed (19ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: true }
 ✔ Element <body> was visible after 56 milliseconds.
LOG     → Completed command waitForElementVisible (59 ms)
INFO Request: GET /wd/hub/session/c063102d2b6f37a80a399192e9005705/title 
 - data:   
 - headers:  {"Accept":"application/json"}
INFO Response 200 GET /wd/hub/session/c063102d2b6f37a80a399192e9005705/title (16ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: 'Google' }
 ✔ Testing if the page title equals "Google".
LOG     → Completed command title (17 ms)
LOG     → Completed command title (16 ms)
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/element 
 - data:  {"using":"css selector","value":"input[type=text]"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":51}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/element (38ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: { ELEMENT: '0.5168266409339404-2' } }
INFO Request: GET /wd/hub/session/c063102d2b6f37a80a399192e9005705/element/0.5168266409339404-2/displayed 
 - data:   
 - headers:  {"Accept":"application/json"}
INFO Response 200 GET /wd/hub/session/c063102d2b6f37a80a399192e9005705/element/0.5168266409339404-2/displayed (34ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: true }
 ✔ Testing if element <input[type=text]> is visible.
LOG     → Completed command isVisible (76 ms)
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/element 
 - data:  {"using":"css selector","value":"input[type=text]"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":51}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/element (19ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: { ELEMENT: '0.5168266409339404-2' } }
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/element/0.5168266409339404-2/value 
 - data:  {"value":["r","e","m","b","r","a","n","d","t"," ","v","a","n"," ","r","i","j","n"]} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":83}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/element/0.5168266409339404-2/value (346ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: null }
LOG     → Completed command setValue (366 ms)
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements 
 - data:  {"using":"css selector","value":"button[name=btnG]"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":52}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements (38ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: [] }
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements 
 - data:  {"using":"css selector","value":"button[name=btnG]"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":52}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements (23ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: [] }
INFO Request: POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements 
 - data:  {"using":"css selector","value":"button[name=btnG]"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":52}
INFO Response 200 POST /wd/hub/session/c063102d2b6f37a80a399192e9005705/elements (67ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: [] }
 ✖ Timed out while waiting for element <button[name=btnG]> to be present for 1000 milliseconds.  - expected "visible" but got: "not found"
    at Object.Demo test Google (/home/example/Projects/example/tests/NightWatch/google.js:9:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)

LOG     → Completed command waitForElementVisible (1133 ms)
INFO Request: DELETE /wd/hub/session/c063102d2b6f37a80a399192e9005705 
 - data:   
 - headers:  {"Content-Length":0}
INFO Response 200 DELETE /wd/hub/session/c063102d2b6f37a80a399192e9005705 (53ms) { sessionId: 'c063102d2b6f37a80a399192e9005705',
  status: 0,
  value: null }
INFO FINISHED
LOG     → Completed command end (56 ms)
LOG     → Completed command session (56 ms)

FAILED:  1 assertions failed and 3 passed (5.724s)

 _________________________________________________

 TEST FAILURE:  1 assertions failed, 3 passed. (5.784s)

 ✖ NightWatch/google

   - Demo test Google (5.724s)
   Timed out while waiting for element <button[name=btnG]> to be present for 1000 milliseconds.  - expected "visible" but got: "not found"
       at Object.Demo test Google (/home/example/Projects/example/tests/NightWatch/google.js:9:8)
       at _combinedTickCallback (internal/process/next_tick.js:131:7)

xurizaemon commented 5 years ago

I was going to submit a test that tests nightwatchjs.org, but I see there's no form interactivity on the site and think that's a significant part of the demo test.

FWIW, I found that replacing <button[name=btnG]> (not on Google AFAICT) with <input[name=btnK]> (visible on Google if I .pause() the test) did not resolve the issue. I'm new to Nightwatch, so maybe this is just my setup / config ... or maybe Google's weird :smiling_imp:

QualityADHD commented 5 years ago

No, what is happening is that when you type in the search input box during the .setValue('input[type=text]', 'rembrandt van rijn') step, a predictive search dropdown appears which then blocks the button from being visible, causing the test to fail. You could work around it by adding browser.Keys.ENTER to just simulate a keyboard enter once you input the text, but eh....

I agree that having an example that works on the home page would be helpful :)

zeachco commented 5 years ago

Why not target a website that doesn't serve different content depending on your region, language, calendar date, and previous searches. I think using Google is a fragile strategy and the demo should target something controlled by the community at least, such as the documentation page of Nightwatchjs for example.

nightwatchjs / nightwatch