Calling .end() when test fails does not halt all tests running

paultitcomb commented 4 years ago

Describe the bug

I am setting _end_session_onfail to true in the nightwatch.json file and it doesn't work. I deliberately put an incorrect selector in the code to introduce an error in the first test run and while it reports an error (Testing if element <Element [name=@signUpCampaignLink]> is visible in 5000ms - expected "is visible" but got: "element could not be located"), it carries on and runs the next test in the series when i would it expect it to exit.

Sample test

The test is in the page object for that specific test and consists of the following (i have taken out all the other functions and elements for clarity):

The @signUpCampaignLink is the one with an incorrect selector.

module.exports = {
    baseUrl: 'https://www.savethechildren.org.uk/how-you-can-help/show-your-support',
    elements: {
        signUpCampaignLink: {
            locateStrategy: 'css selector',
            selector: '#formcontainerzzz__1'
        }
    },
    commands: [
        require('../modules/commonCommands'),
        {
            // Click on the 'Sign up to campaign with us' link at the bottom of the page
            clickCampaignLink: function () {
                this.pause(1000)
                    .verify.visible('@signUpCampaignLink')
                    .moveToElement('@signUpCampaignLink', 10, 10)
                    .waitForElementVisible('@signUpCampaignLink', 500)
                    .click('@signUpCampaignLink');
                return this;
            }
        }
    ]
};

Verbose output

debug.log

```txt Connecting to localhost on port 4444... Request POST /wd/hub/session { desiredCapabilities: { browserName: 'chrome', platform: 'ANY', javascriptEnabled: true, acceptSslCerts: true, chromeOptions: { w3c: false, args: [ '--headless', '--no-sandbox', '--window-size=1920,1080', '--no-gpu', '--disable-dev-shm-usage' ] }, name: 'Data Forms/Campaign With Us Test' } ⠹ Connecting to localhost on port 4444... Response 200 POST /wd/hub/session (1047ms) { status: 0, sessionId: 'd8699a514b4051bf662cb958ef8a9848', value: { acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: 'chrome', chrome: { chromedriverVersion: '83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103@{#416})', userDataDir: '/var/folders/0v/mk4y6g694956vp48h9nk19h80000gp/T/.com.google.Chrome.cJ9HVC' }, cssSelectorsEnabled: true, databaseEnabled: false, 'goog:chromeOptions': { debuggerAddress: 'localhost:49480' }, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: 'normal', platform: 'Mac OS X', proxy: {}, rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: { implicit: 0, pageLoad: 300000, script: 30000 }, unexpectedAlertBehaviour: 'ignore', version: '83.0.4103.61', webStorageEnabled: true, 'webauthn:virtualAuthenticators': true, 'webdriver.remote.sessionid': 'd8699a514b4051bf662cb958ef8a9848' } ℹ Connected to localhost on port 4444 (1074ms). Using: chrome (83.0.4103.61) on Mac OS X platform. Received session with ID: d8699a514b4051bf662cb958ef8a9848 → Running [before]: → Completed [before]. Running: Campaign with us data form submission → Running [beforeEach]: → Completed [beforeEach]. → Running command: url ('https://savethechildren-preprod-65-uk.adobecqms.net/', ) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/url { url: 'https://savethechildren-preprod-65-uk.adobecqms.net/' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/url (1667ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: null } → Completed command: url ('https://savethechildren-preprod-65-uk.adobecqms.net/', ) (1669ms) → Running command: waitForElementVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 5000) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '.ui-navigation-desk' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (142ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.9119150479145566-1' } ] } Request GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-1/displayed Response 200 GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-1/displayed (71ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: true } ✔ Element <.ui-navigation-desk> was visible after 217 milliseconds. → Completed command: waitForElementVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 5000) (219ms) → Running command: pause (2000) → Completed command: pause (2000) (2006ms) → Running command: element ('css selector', '#pdv4wrap', [Function]) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element { using: 'css selector', value: '#pdv4wrap' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element (20ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 7, value: { message: 'no such element: Unable to locate element: {"method":"css selector","selector":"#pdv4wrap"}', error: [ ' (Session info: headless chrome=83.0.4103.61)', ' (Driver info: chromedriver=83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103@{#416}),platform=Mac OS X 10.13.6 x86_64)' ] } } → Completed command: element ('css selector', '#pdv4wrap', [Function]) (22ms) → Running command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) → Running command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: 'button[data-index = "2"]' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (13ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.9119150479145566-2' } ] } Request GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-2/displayed Response 200 GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-2/displayed (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: true } ✔ Testing if element is visible (28ms) → Completed command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) (30ms) → Completed command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) (27ms) → Running command: click ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: 'button[data-index = "2"]' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (8ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.9119150479145566-2' } ] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-2/click {} Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-2/click (71ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: null } → Completed command: click ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) (79ms) → Running command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) → Running command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: 'a[href="/how-you-can-help"]' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (13ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.9119150479145566-3' }, { ELEMENT: '0.9119150479145566-4' }, { ELEMENT: '0.9119150479145566-5' }, { ELEMENT: '0.9119150479145566-6' } ] } Warning: More than one element (4) found for element with selector: "a[href="/how-you-can-help"]". Only the first one will be used. Request GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-3/displayed Response 200 GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-3/displayed (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: true } ✔ Testing if element is visible (26ms) → Completed command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) (28ms) → Completed command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) (27ms) → Running command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) → Running command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: 'a[href="/how-you-can-help/show-your-support"]' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.9119150479145566-7' }, { ELEMENT: '0.9119150479145566-8' } ] } Warning: More than one element (2) found for element with selector: "a[href="/how-you-can-help/show-your-support"]". Only the first one will be used. Request GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-7/displayed Response 200 GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-7/displayed (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: true } ✔ Testing if element is visible (25ms) → Completed command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) (25ms) → Completed command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) (24ms) → Running command: click ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: 'a[href="/how-you-can-help/show-your-support"]' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (8ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.9119150479145566-7' }, { ELEMENT: '0.9119150479145566-8' } ] } Warning: More than one element (2) found for element with selector: "a[href="/how-you-can-help/show-your-support"]". Only the first one will be used. Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-7/click {} Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.9119150479145566-7/click (70ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: null } → Completed command: click ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) (78ms) → Running command: pause (5000) → Completed command: pause (5000) (5002ms) → Running command: waitForElementVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 10000) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '.ui-navigation-desk' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (19ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [ { ELEMENT: '0.19922745056773183-1' } ] } Request GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.19922745056773183-1/displayed Response 200 GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/element/0.19922745056773183-1/displayed (14ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: true } ✔ Element <.ui-navigation-desk> was visible after 35 milliseconds. → Completed command: waitForElementVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 10000) (36ms) → Running command: pause (1000) → Completed command: pause (1000) (1004ms) → Running command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) → Running command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (16ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (9ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (10ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } ✖ Testing if element is visible in 5000ms - expected "is visible" but got: "element could not be located" (5166ms) at Page.clickCampaignLink (/Users/paul/automated-gui-testing/pages/campaignWithUsPage.js:61:14) at Object.Campaign with us data form submission (/Users/paul/automated-gui-testing/tests/dataForms/campaignWithUsTest.js:15:5) → Completed command: isVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, [Function]) (5165ms) → Completed command: verify.visible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}) (5167ms) → Running command: moveToElement ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 10, 10) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (8ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (10ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (10ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (11ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (12ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } → Completed command: moveToElement ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 10, 10) (5171ms) → Running command: waitForElementVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 500) Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (8ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } Request POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements { using: 'css selector', value: '#formcontainerzzz__1' } Response 200 POST /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/elements (13ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: [] } ✖ Timed out while waiting for element <#formcontainerzzz__1> to be present for 500 milliseconds. - expected "visible" but got: "not found" (528ms) at Page.clickCampaignLink (/Users/paul/automated-gui-testing/pages/campaignWithUsPage.js:63:7) at Object.Campaign with us data form submission (/Users/paul/automated-gui-testing/tests/dataForms/campaignWithUsTest.js:15:5) → Completed command: waitForElementVisible ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout, retryInterval, message}, 500) (529ms) → Running [afterEach]: → Running command: end ([Function]) Failures in "Campaign with us data form submission". Taking screenshot... → Running command: saveScreenshot ('/Users/paul/automated-gui-testing/screenshots/dataForms/campaignWithUsTest/Campaign-with-us-data-form-submission_FAILED_May-26-2020-152025-GMT+0100-(British-Summer.png', [Function]) → Running command: screenshot (false, [Function]) Request GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/screenshot Response 200 GET /wd/hub/session/d8699a514b4051bf662cb958ef8a9848/screenshot (2047ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: '', suppressBase64Data: true } → Completed command: screenshot (false, [Function]) (2406ms) → Completed command: saveScreenshot ('/Users/paul/automated-gui-testing/screenshots/dataForms/campaignWithUsTest/Campaign-with-us-data-form-submission_FAILED_May-26-2020-152025-GMT+0100-(British-Summer.png', [Function]) (2425ms) → Running command: session ('delete', [Function]) Request DELETE /wd/hub/session/d8699a514b4051bf662cb958ef8a9848 Response 200 DELETE /wd/hub/session/d8699a514b4051bf662cb958ef8a9848 (55ms) { sessionId: 'd8699a514b4051bf662cb958ef8a9848', status: 0, value: null } → Completed [afterEach]. FAILED: 2 assertions failed, 2 errors and 5 passed (23.584s) → Completed command: end ([Function]) (2488ms) → Completed command: session ('delete', [Function]) (62ms) → Running [after]: → Completed [after]. ```

Configuration

nightwatch.json

Here is the dataForms.conf.js file ```js module.exports = (function (settings) { settings.selenium.server_path = require('selenium-server-standalone-jar').path; settings.selenium.cli_args['webdriver.chrome.driver'] = require('chromedriver').path; settings.end_session_on_fail = true; settings.skip_testcases_on_fail = true; return settings; })(require('../nightwatch.json')); ``` Here is the nightwatch.json ```json { "src_folders": ["tests"], "output_folder": "reports", "custom_commands_path": "", "custom_assertions_path": "", "page_objects_path": "pages", "globals_path": "", "end_session_on_fail": false, "skip_testcases_on_fail": false, "selenium": { "start_process": true, "server_path": "./node_modules/selenium-server-standalone-jar/jar/selenium-server-standalone-3.141.59.jar", "log_path": "", "port": 4444, "check_process_delay": 5000, "cli_args": { "webdriver.chrome.driver": "~/usr/bin/chromedriver", "webdriver.gecko.driver": "", "webdriver.edge.driver": "" } }, "test_settings": { "default": { "launch_url": "https://www.savethechildren.org.uk/", "selenium_port": 4444, "selenium_host": "localhost", "silent": true, "screenshots": { "enabled": true, "path": "./screenshots", "on_failure": true, "on_error": true }, "desiredCapabilities": { "browserName": "chrome", "javascriptEnabled": true, "acceptSslCerts": true, "chromeOptions": { "w3c": false, "args": [ "--headless", "--no-sandbox", "--window-size=1920,1080", "--no-gpu", "--disable-dev-shm-usage" ] } }, "request_timeout_options": { "timeout": 300000, "retry_attempts": 5 } }, "prod": { "launch_url": "https://www.savethechildren.org.uk/" } } } ```

Your Environment

Executable	Version
`nightwatch --version`	1.3.5
`npm --version`	6.13.4
`node --version`	v12.16.0

Browser driver	Version
chromedriver	83.0.0

OS	Version
macOS High Sierra	10.13.6

beatfactor commented 4 years ago

I'm afraid you're interpreting this setting incorrectly. The "session" here refers to the Selenium/Webdriver session (i.e. the browser window), not the entire test run, and it is true by default.

It is only useful to disable it if you intend on doing some extra logging or custom processing (e.g. take a screenshot) after a test failure, after which you need to close the session yourself (by calling .end().

paultitcomb commented 4 years ago

Fair enough i take your point on the _end_session_onfail i misunderstood what that was for, but calling .end() does not work either.

Here is my code:

module.exports = {
    tags: ['dataForms'],
    'Campaign with us data form submission': function (browser) {
        browser.page
            .homePage()
            .navigate()
            .verifyOnPage()
            .closePopup()
            .clickHowYouCanHelpButton()
            .verifyOnHowYouCanHelpFlyout()
            .clickShowSupportLink();
        browser.page
            .campaignWithUsPage()
            .verifyOnPage()
            .clickCampaignLink()
            .fillInCampaignForm(browser)
            .verifyThankYouMessage()
            .end();
    },
    afterEach: function (browser, done) {
        if (browser.currentTest.results.errors) {
            done();
            browser.end();
        }
    }
};

I would expect that to stop all tests running if there is an error - it doesn't.

Topperfalkon commented 4 years ago

Isn't skip_testcases_on_fail what you're looking for?

https://nightwatchjs.org/gettingstarted/configuration/#test-session-settings

beatfactor commented 4 years ago

Isn't skip_testcases_on_fail what you're looking for?

https://nightwatchjs.org/gettingstarted/configuration/#test-session-settings

That will only skip the test cases in the current test suite, and it's also true by default.

Topperfalkon commented 4 years ago

Ah, right. I misunderstood

nightwatchjs / nightwatch