search

nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.83k stars 1.32k forks source link

Global expect node execution is leaking into other subsequent nodes. #3751

Open reallymello opened 1 year ago

reallymello commented 1 year ago

Description of the bug/issue

When I use browser.expect on an element found using .find() I expect it to run my expect assertion, but instead the browser terminates and I get an unhandled rejection.

image

Steps to reproduce

Running this test without line 18 works, but with the browser seems to terminate early

import { NightwatchTests } from 'nightwatch';

const home: NightwatchTests = {
  'Shadow root test': async () => {
    browser
      .navigateTo(
        'https://mdn.github.io/web-components-examples/popup-info-box-web-component/'
      )
      .waitForElementVisible('form');

    const shadowRootEl = await browser.getShadowRoot('popup-info');
    const infoElement = await shadowRootEl.find('.info');

    expect(infoElement.property('innerHTML')).to.include(
      'card validation code'
    );

    browser.expect.element(infoElement).text.to.contain('card'); // Line 18
  },
};

export default home;

Sample test

import { NightwatchTests } from 'nightwatch';

const home: NightwatchTests = {
  'Shadow root test': async () => {
    browser
      .navigateTo(
        'https://mdn.github.io/web-components-examples/popup-info-box-web-component/'
      )
      .waitForElementVisible('form');

    const shadowRootEl = await browser.getShadowRoot('popup-info');
    const infoElement = await shadowRootEl.find('.info');

    expect(infoElement.property('innerHTML')).to.include(
      'card validation code'
    );

    browser.expect.element(infoElement).text.to.contain('card');
  },
};

export default home;

Command to run

npx nightwatch

Verbose Output

npx nightwatch --verbose
 Now you can run TS tests directly using Nightwatch.
 Launching up to 1 concurrent test worker processes...

 Running:  default: google.ts 

DevTools listening on ws://127.0.0.1:56784/devtools/browser/9d2c7e3f-675a-4f9a-893b-928ba00eaf0c

×  default: google.ts   Now you can run TS tests directly using Nightwatch. 

 [Google] Test Suite 
 ────────────────────────────────────── 
 Starting ChromeDriver with server_path=C:\Projects\nightwatchTutorials\nw30typescriptExample\node_modules\chromedriver\lib\chromedriver\chromedriver.exe... 
   Request POST /session   
 { 
      capabilities: { 
        firstMatch: [ {} ], 
        alwaysMatch: { browserName: 'chrome', 'goog:chromeOptions': {} } 
      } 
   } 
   Response 200 POST /session (535ms) 
 { 
      value: { 
        capabilities: { 
          acceptInsecureCerts: false, 
          browserName: 'chrome', 
          browserVersion: '113.0.5672.127', 
          chrome: { 
            chromedriverVersion: '113.0.5672.63 (0e1a4471d5ae5bf128b1bd8f4d627c8cbd55f70c-refs/branch-heads/5672@{#912})', 
            userDataDir: 'C:\\Users\\Mr\\AppData\\Local\\Temp\\scoped_dir10120_2012294010' 
          }, 
          'goog:chromeOptions': { debuggerAddress: 'localhost:56784' }, 
          networkConnectionEnabled: false,
          pageLoadStrategy: 'normal',
          platformName: 'windows',
          proxy: {},
          setWindowRect: true,
          strictFileInteractability: false,
          timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
          unhandledPromptBehavior: 'dismiss and notify',
          'webauthn:extension:credBlob': true,
          'webauthn:extension:largeBlob': true,
          'webauthn:extension:minPinLength': true,
          'webauthn:extension:prf': true,
          'webauthn:virtualAuthenticators': true
        },
        sessionId: '51b9a5dcb497f4e84598dfd20d016fd7'
      }
   }
 Using: chrome (113.0.5672.127) on WINDOWS.
 Received session with ID: 51b9a5dcb497f4e84598dfd20d016fd7 

 → Running [before]:
 → Completed [before].

 – Shadow root test
 → Running [beforeEach]:
 → Completed [beforeEach].

  → Running command: navigateTo ('https://mdn.github.io/web-components-examples/popup-info-box-web-component/') 
   Request POST /session/51b9a5dcb497f4e84598dfd20d016fd7/url  
 {
      url: 'https://mdn.github.io/web-components-examples/popup-info-box-web-component/'
   }
   Response 200 POST /session/51b9a5dcb497f4e84598dfd20d016fd7/url (155ms)
 { value: null }
  → Completed command: navigateTo ('https://mdn.github.io/web-components-examples/popup-info-box-web-component/') (158ms)

  → Running command: waitForElementVisible ('form') 
   Request POST /session/51b9a5dcb497f4e84598dfd20d016fd7/elements  
 { using: 'css selector', value: 'form' }
   Response 200 POST /session/51b9a5dcb497f4e84598dfd20d016fd7/elements (13ms)
 {
      value: [
        {
          'element-6066-11e4-a52e-4f735466cecf': '4B973618E217399C036DD775C117FFC8_element_4'
        }
      ]
   }
   Request POST /session/51b9a5dcb497f4e84598dfd20d016fd7/execute/sync  
 {
      script: 'return (function(){return (function(){var k=this||self;function aa(a){return"string"==typeof a}function ba(a,b){a=a.split(".");var c=k;a[0]in c||"undefined"==typeof c.execScript||c.execScript("var "+a... (44027 characters)',
      args: [
        {
          'element-6066-11e4-a52e-4f735466cecf': '4B973618E217399C036DD775C117FFC8_element_4',
          ELEMENT: '4B973618E217399C036DD775C117FFC8_element_4'
        }
      ]
   }
   Response 200 POST /session/51b9a5dcb497f4e84598dfd20d016fd7/execute/sync (19ms)
 { value: true }
 √ Element <form> was visible after 35 milliseconds.
  → Completed command: waitForElementVisible ('form') (37ms)

  → Running command: getShadowRoot ('popup-info') 
   Request POST /session/51b9a5dcb497f4e84598dfd20d016fd7/elements  
 { using: 'css selector', value: 'popup-info' }
   Response 200 POST /session/51b9a5dcb497f4e84598dfd20d016fd7/elements (10ms)
 {
      value: [
        {
          'element-6066-11e4-a52e-4f735466cecf': '4B973618E217399C036DD775C117FFC8_element_5'
        }
      ]
   }
   Request POST /session/51b9a5dcb497f4e84598dfd20d016fd7/execute/sync  
 {
      script: 'return (function(element) {\n' +
        '            return element && element.shadowRoot;\n' +
        '          }).apply(null, arguments);... (114 characters)',
      args: [
        {
          'element-6066-11e4-a52e-4f735466cecf': '4B973618E217399C036DD775C117FFC8_element_5'
        }
      ]
   }
   Response 200 POST /session/51b9a5dcb497f4e84598dfd20d016fd7/execute/sync (6ms)
 {
      value: {
        'shadow-6066-11e4-a52e-4f735466cecf': '4B973618E217399C036DD775C117FFC8_element_6'
      }
   }
  → Completed command: getShadowRoot ('popup-info') (20ms)

  → Running command: element().find ('.info') 
   Request POST /session/51b9a5dcb497f4e84598dfd20d016fd7/element/4B973618E217399C036DD775C117FFC8_element_6/element  
 { using: 'css selector', value: '.info' }
   Response 200 POST /session/51b9a5dcb497f4e84598dfd20d016fd7/element/4B973618E217399C036DD775C117FFC8_element_6/element (10ms)
 {
      value: {
        'element-6066-11e4-a52e-4f735466cecf': '4B973618E217399C036DD775C117FFC8_element_7'
      }
   }
  → Completed command: element().find ({value, using}) (12ms)
 → Running [afterEach]:

  → Running command: element().property ('innerHTML') 
   Request GET /session/51b9a5dcb497f4e84598dfd20d016fd7/element/4B973618E217399C036DD775C117FFC8_element_7/property/innerHTML  

   Response 200 GET /session/51b9a5dcb497f4e84598dfd20d016fd7/element/4B973618E217399C036DD775C117FFC8_element_7/property/innerHTML (6ms)
 {
      value: 'Your card validation code (CVC) is an extra security feature — it is the last 3 or 4 numbers on the ...',
      suppressBase64Data: true
   }
  → Completed command: element().property ('innerHTML') (8ms)

  → Running command: expect.element ({name, webElement, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}) 
   Request GET /session/51b9a5dcb497f4e84598dfd20d016fd7/element/4B973618E217399C036DD775C117FFC8_element_7/text  

  → Running command: expect(<value>)..to.include('card validation code') () 
 √ Expected 'Your card validation code (CVC) is an…'  to include('card validation code'):
  → Completed command: expect(<value>)..to.include('card validation code') () (2ms)
   Response 200 GET /session/51b9a5dcb497f4e84598dfd20d016fd7/element/4B973618E217399C036DD775C117FFC8_element_7/text (15ms)
 { value: '' }
  → Completed command: expect.element ({name, webElement, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}) (38ms) 
 → Completed [afterEach].
 √ default: google.ts [Google] Shadow root test (306ms)
 → Running [after]:
 → Completed [after].

  → Running command: end (true) 

  → Running command: session ('delete', [Function]) 
   Request DELETE /session/51b9a5dcb497f4e84598dfd20d016fd7  
   Response 200 DELETE /session/51b9a5dcb497f4e84598dfd20d016fd7 (38ms)
 { value: null }
  → Completed command: end (true) (43ms)
  → Completed command: session ('delete', [Function]) (41ms)
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
    Error   Error while running .getElementText() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used. 

  Error
    Unexpected non-string or array result value returned for contains: null
  ✖ NightwatchAssertError
    Expected element <web element{4B973618E217399C036DD775C117FFC8_element_7}> text to contain: "card" - expected "contain 'card'" but got: "null" (5134ms)

     Error location: 
     C:\Projects\nightwatchTutorials\nw30typescriptExample\nightwatch\tests\google.ts:
     ––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
      16 |     );
      17 |
      18 |     browser.expect.element(infoElement).text.to.contain('card'); 
      19 |   },
      20 | };
     ––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

  Error
    unhandledRejection: Expected element <web element{4B973618E217399C036DD775C117FFC8_element_7}> text to contain: "card" - expected "contain 'card'" but got: "null" (5134ms)
     at Object.Shadow root test (C:\Projects\nightwatchTutorials\nw30typescriptExample\nightwatch\tests\google.ts:18:20)

Nightwatch Configuration

// Refer to the online docs for more details:
// https://nightwatchjs.org/gettingstarted/configuration/
//

//  _   _  _         _      _                     _          _
// | \ | |(_)       | |    | |                   | |        | |
// |  \| | _   __ _ | |__  | |_ __      __  __ _ | |_   ___ | |__
// | . ` || | / _` || '_ \ | __|\ \ /\ / / / _` || __| / __|| '_ \
// | |\  || || (_| || | | || |_  \ V  V / | (_| || |_ | (__ | | | |
// \_| \_/|_| \__, ||_| |_| \__|  \_/\_/   \__,_| \__| \___||_| |_|
//             __/ |
//            |___/

module.exports = {
  // An array of folders (excluding subfolders) where your tests are located;
  // if this is not specified, the test source must be passed as the second argument to the test runner.
  src_folders: ['nightwatch/tests'],

  // See https://nightwatchjs.org/guide/concepts/page-object-model.html
  page_objects_path: ['nightwatch/page-objects'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-custom-commands.html
  custom_commands_path: ['nightwatch/commands'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-custom-assertions.html
  custom_assertions_path: [],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-plugins.html
  plugins: [],

  // See https://nightwatchjs.org/guide/concepts/test-globals.html
  globals_path: '',

  webdriver: {},

  test_workers: {
    enabled: true,
  },

  test_settings: {
    default: {
      disable_error_log: false,
      launch_url: 'http://localhost',

      screenshots: {
        enabled: false,
        path: 'screens',
        on_failure: true,
      },

      desiredCapabilities: {
        browserName: 'chrome',
      },

      webdriver: {
        start_process: true,
        server_path: '',
      },
    },

    chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        'goog:chromeOptions': {
          // More info on Chromedriver: https://sites.google.com/a/chromium.org/chromedriver/
          //
          // w3c:false tells Chromedriver to run using the legacy JSONWire protocol (not required in Chrome 78)
          w3c: true,
          args: [
            //'--no-sandbox',
            //'--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            //'--headless'
          ],
        },
      },

      webdriver: {
        start_process: true,
        server_path: '',
        cli_args: [
          // --verbose
        ],
      },
    },
  },
};

Nightwatch.js Version

2.6.21

Node Version

18.16.0

Browser

Chrome 113

Operating System

Windows 10

Additional Information

image

gravityvi commented 1 year ago

The element has been found, but it results in unhandled rejection while running assertions for text.

gravityvi commented 1 year ago

The root cause is the global expect node leaking into subsequent nightwatch command nodes.

Example:

   expect(infoElement.property('innerHTML')).to.include(
      'card validation code'
    );

    browser.click(infoElement);

For these two nightwatch commands here are the verbose logs

Screenshot 2023-05-31 at 6 15 54 PM

Logs show that the global expect command is executed in between click command execution