nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.79k stars 1.31k forks source link

Nightwatch stops execution without any warning if callback of the command function of a custom assertion is never run #3951

Closed kumarchandresh closed 11 months ago

kumarchandresh commented 11 months ago

Description of the bug/issue

When the callback of the command function of a custom assertion is never called,

// Example of a custom assertion
exports.assertion = {
    // ...
    this.command = function (callback) {
        // What happens if I forget to call the callback,
        // or place it inside a conditional that never run?
    }
    // ...
}

I would expect Nighwatch to warn me (maybe through some TimeoutError or something), but Nightwatch stops execution immediately and doesn't report any error/warning.

Steps to reproduce

  1. Clone repo https://github.com/kumarchandresh/nightwatch-issue-2
  2. Execute node nightwatch tests/good.test.js
  3. Execute node nightwatch tests/bad.test.js
  4. See bad.test.js stops execution and does not run completely whereas good.test.js executes as expected.

Sample test

No response

Command to run

No response

Verbose Output

No response

Nightwatch Configuration

No response

Nightwatch.js Version

3.2.1

Node Version

18.17.0

Browser

Chrome 118.0.0

Operating System

Windows 11

Additional Information

Tested on v2.6.20 and v3.2.1

AutomatedTester commented 11 months ago

Unfortunately there is nothing we can do here as we are limited by what we can use in JavaScript.

DovieW commented 7 months ago

@AutomatedTester I'm seeing a similar issue I think: image

I see that Nightwatch still has some output at the end. This makes me think that it is possible for Nightwatch to create a report file specifying the error.

How would I handle this situation otherwise? During our CI/CD runs, I would have to wrap the npx nightwatch command to handle JS errors I assume.