nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.83k stars 1.33k forks source link

Callback function is not called if there is an error in the command #2255

Closed ruben0626 closed 4 years ago

ruben0626 commented 4 years ago

Describe the bug

this.callback is not invoked if there is an error in the command execution https://github.com/nightwatchjs/nightwatch/blob/master/lib/api/element-commands/_baseElementCommand.js#L55:#L60

I have many custom commands that use the click callback to execute extra steps, now those commands are broken because the callback is not executed if the click fails (i.e., if the element is present but is not clickable).

A possible solution: https://github.com/nightwatchjs/nightwatch/blob/master/lib/api/element-commands/_baseElementCommand.js#L55:#L60

errorHandler: (err) => {
          if (typeof this.callback === 'function') {
            return this.complete(err, err.response);
          }

          if (err.name !== 'TimeoutError') {
            return err;
          }

          return this.handleElementError(err);
        }

Sample test

sampleTest.js

```js // Please add the sample test here module.exports = { 'Click callback should not be ignored': (client) => { client.url('http://example.com', () => { client.execute(function() { // Move the element outside the viewport document.querySelector('body').style.paddingLeft = '99999999px'; }, () => { client.click('h1', (result) => { // This callback won't be executed console.log('callback', result); }); }); }); }, } ```

Run with command

$ nightwatch test/sampleTest.js --your-other-arguments-here

Verbose output

debug.log

```txt Starting ChromeDriver on port 9515... ChromeDriver up and running on port 9515 with pid: 79536 (155ms). [INFO] [Nightwatch Extra] Found nightwatch configuration at /Users/rjimenez/Development/walmart/walmart-test-framework/packages/demo/src/nightwatch.json [Nw Extra Funcspec] Test Suite ============================== ⠋ Connecting to on port 9515... Request POST http://undefined:9515 /session { desiredCapabilities: { browserName: 'chrome', platform: 'ANY', chromeOptions: { args: [ '--start-maximized', '--start-fullscreen', 'auto-open-devtools-for-tabs' ], mobileEmulation: { deviceMetrics: [Object] } }, ⠋ Connecting to on port 9515... Response 200 POST http://undefined:9515/session (1658ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: { acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: 'chrome', chrome: { chromedriverVersion: '78.0.3904.70 (edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800})', userDataDir: '/var/folders/zv/2_hr3c2j4gs05nsxxvx_pfm00000gn/T/.com.google.Chrome.DUkf0J' }, cssSelectorsEnabled: true, databaseEnabled: false, 'goog:chromeOptions': { debuggerAddress: 'localhost:62690' }, handlesAlerts: true, hasTouchScreen: true, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: true, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: 'normal', platform: 'Mac OS X', proxy: {}, rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: { implicit: 0, pageLoad: 300000, script: 30000 }, unexpectedAlertBehaviour: 'ignore', version: '78.0.3904.108', ℹ Connected to on port 9515 (1710ms). Using: chrome (78.0.3904.108) on Mac OS X platform. Received session with ID: 90c5f838c7d047b2178b040bcfd4fd29 → Running [before]: → Completed [before]. Running: Click callback should not be ignored → Running [beforeEach]: → Running command: timeoutsAsyncScript (5000) Request POST http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29/timeouts/async_script { ms: 5000 } Response 200 POST http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29/timeouts/async_script (2ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: null } → Completed command: timeoutsAsyncScript (5000) (5ms) → Completed [beforeEach]. → Running command: url ('http://example.com', [Function]) Request POST http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29/url { url: 'http://example.com' } Response 200 POST http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29/url (424ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: null } → Completed command: url ('http://example.com', [Function]) (426ms) → Running command: execute ([Function], [Function]) Request POST http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29/execute { script: 'var passedArgs = Array.prototype.slice.call(arguments,0); return function() {\n // Move the element outside the viewport\n document.querySelector(\'body\').style.paddingLeft = \'99999999px\';\n }.apply(window, passedArgs);', args: [] } Response 200 POST http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29/execute (101ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: null } → Completed command: execute ([Function], [Function]) (102ms) → Running command: click ('h1', [Function]) Request POST http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29/elements { using: 'css selector', value: 'h1' } Response 200 POST http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29/elements (21ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: [ { ELEMENT: '0.6006250174841665-1' } ] } Request POST http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29/element/0.6006250174841665-1/click {} Response 200 POST http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29/element/0.6006250174841665-1/click (72ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 64, value: { message: 'element click intercepted: Element is not clickable at point (5419, 152)', error: [ ' (Session info: chrome=78.0.3904.108)', ' (Driver info: chromedriver=78.0.3904.70 (edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800}),platform=Mac OS X 10.14.6 x86_64)' ] } } Error while running .clickElement() protocol action: element click intercepted: Element is not clickable at point (5419, 152) → Completed command: click ('h1', [Function]) (112ms) → Running [afterEach]: → Completed [afterEach]. FAILED: 1 errors (651ms) → Running [after]: → Running command: end () Failures in "Click callback should not be ignored". Taking screenshot... → Completed [after]. → Running command: end ([Function]) Failures in "Click callback should not be ignored". Taking screenshot... → Running command: saveScreenshot ('/Users/rjimenez/Development/walmart/walmart-test-framework/packages/demo/reports/nw-extra.funcspec/Click-callback-should-not-be-ignored_FAILED_Nov-22-2019-180433-GMT-0400-(Atlantic-Standard.png', [Function]) → Running command: screenshot (false, [Function]) Request GET http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29/screenshot Response 200 GET http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29/screenshot (693ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: '', suppressBase64Data: true } → Completed command: screenshot (false, [Function]) (699ms) → Completed command: saveScreenshot ('/Users/rjimenez/Development/walmart/walmart-test-framework/packages/demo/reports/nw-extra.funcspec/Click-callback-should-not-be-ignored_FAILED_Nov-22-2019-180433-GMT-0400-(Atlantic-Standard.png', [Function]) (706ms) → Running command: session ('delete', [Function]) Request DELETE http://undefined:9515 /session/90c5f838c7d047b2178b040bcfd4fd29 Response 200 DELETE http://undefined:9515/session/90c5f838c7d047b2178b040bcfd4fd29 (54ms) { sessionId: '90c5f838c7d047b2178b040bcfd4fd29', status: 0, value: null } → Completed command: end ([Function]) (772ms) → Completed command: session ('delete', [Function]) (63ms) _________________________________________________ TEST FAILURE: 1 error during execution; 0 tests failed, 0 passed (3.831s) ✖ nw-extra.funcspec – Click callback should not be ignored (651ms) TimeoutError: An error occurred while running .click() command on

: {"status":-1,"state":"","value":{"message":"element click intercepted: Element is not clickable at point (5419, 152)","error":[" (Session info: chrome=78.0.3904.108)"," (Driver info: chromedriver=78.0.3904.70 (edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800}),platform=Mac OS X 10.14.6 x86_64)"]},"errorStatus":64,"error":"element click intercepted: Element is not clickable at point (5419, 152)","httpStatusCode":200} at process._tickCallback (internal/process/next_tick.js:68:7) Wrote report file to: reports/CHROME_78.0.3904.108_Mac_OS_X_nw-extra.funcspec.xml. Wrote log file to: /Users/rjimenez/Development/walmart/walmart-test-framework/packages/demo/chromedriver.log. ChromeDriver process closed. error Command failed with exit code 5. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. Process finished with exit code 5 ```

Configuration

nightwatch.json

```js { "your": { "config": "here" } } ```

Your Environment

Executable Version
nightwatch --version 1.3.0
npm --version 6.12.1
yarn --version 1.17.3
node --version 10.16.3

| Browser driver | Version | | chromedriver | 76 |

| OS | Version | | Windows 10 | 1607 |

beatfactor commented 4 years ago

@ruben0626 which version are you upgrading from? this isn't a new change.

Also, you can better write the test in this way, there's no need for nested callbacks:

module.exports = {
  'Click callback should not be ignored': (client) => {
    client
      .url('http://example.com')
      .execute(function() {
        // Move the element outside the viewport
        document.querySelector('body').style.paddingLeft = '99999999px';
      })
      .click('h1', (result) => {
        // This callback won't be executed
        console.log('callback', result);
      });
    }
};
beatfactor commented 4 years ago

@phaseen can you please use the Mailing List or to ask for assistance? Your question is not relevant to the issue at hand here.

ruben0626 commented 4 years ago

Thanks @beatfactor , I am upgrading from v0.9.

ruben0626 commented 4 years ago

@beatfactor https://github.com/nightwatchjs/nightwatch/commit/7617ab8fb8c6554fdd20218a689e6a6bbffb2142 did fix the issue about the callback not being executed but the test is marked as failed anyway because of the not clickable element. In NW 0.9 the same test is passing OK. Is this expected for NW 1.x ?

image

beatfactor commented 4 years ago

Yeah, it's expected to fail for all errors. You can only suppress errors about the element not being found, but in your case the error is different.

I think the ideal behaviour here would be to use es6 async/await and catch the error locally, but the try/catch doesn't work yet in the current version (1.3.1).

module.exports = {
  'Click callback should not be ignored': async client => {
     try {
      await client.click('h1', (result) => {
        // This callback won't be executed
      });
    } catch (err) {
      console.error('ERROR', err)
    }
  }
};
ruben0626 commented 4 years ago

@beatfactor Nice! I will be waiting for that feature :) Thanks!

JoseInTheArena commented 4 years ago

@beatfactor is this something you'll be working on for the next version?

beatfactor commented 4 years ago

@jarzuaga Yes, it's on my mind for a while.

kretschmannj commented 3 years ago

@beatfactor Just wondering the status on this?