[Enhancement] Take screenshot for every failed assertion and test case

[pawlakmaly]

Is your feature request related to a problem? Please describe.

Recently I've integrated my tests on Azure Pipelines with publishing the reports. I've use nunit3 reporter so the reports contains attachments like screenshots of fails. But recently I found out that for the whole test suit only one screenshot is made for the last test case if it fails (If the last test cases is not failed, no screenshot is done event if all previous test cases failed - reported as separate bug). It's a bit useless when i use flag to not abort rest of the test cases on any fail. Also if test cases contains several assertion only one screen is made.

Describe the solution you'd like

I would like to have screenshots of every failed test case in suit, not only the last one. And also I would like to have option to have screenshot of every failed assertion so i could see in Azure report what went wrong.

Describe how the solution will work

As an configuration option Nightwatch should have possibility to make screenshot for every test case in suite or every assertion and attached it to test result so the report would contain screenshot of every fail(test case or assertion).

Describe alternatives you've considered

Or at least possibility to add screenshot to assertion result object.

[mkelly2000]

To take screenshots for each failed test cases you need to close the browser session after it fails (this is why the last test case takes the screenshot)
Im not sure if this will solve all your issues above but should take at least 1 screenshot for each case

afterEach: function(browser) { browser.end(); },

[pawlakmaly]

I know I can do that, but in my case it's not a solution as my test suit contains multiple test cases or test cases contains multiple assertions. Application that I'm testing requires log in and I'm using UI to do that and it usually takes about 7 seconds so my test suit would spend twice as much time on logging in than on doing actual tests.

[gravityvi]

This is a duplicate issue of #2402 which is fixed 🎉

[pawlakmaly]

Why this is closed ? Version 1.6.4 is still makin only one screenshot for whole test suit @beatfactor

[Dharin-shah]

@pawlakmaly - can you share an example? And please raise a new issue in correct format as specified by guidelines

[beatfactor]

Actually this is not a duplicate of #2402, it's a new feature.

[kretschmannj]

Based on comments/suggestions from other posts on this subject, I coded a workaround that basically checks for any failures after each test case is run and if a failure is found then the browser session is closed. This allows Nightwatch to capture a screenshot. In order to continue the test, I have a beforeEach that checks to see if a new session was started and executes additional actions as necessary. Here's a snippet of the code ...

    before : function (browser) {
        browser.globals.sessionId = browser.sessionId
        browser.url(unsecuredurl)
    },

    beforeEach: function (browser) {
        browser.globals.newSession = false
        if (browser.sessionId != browser.globals.sessionId) {
            browser.assert.log(true, '*** A new session was started \(you may need to handle navigation to the proper starting location\)')
            browser.globals.sessionId = browser.sessionId
            browser.globals.newSession = true
            browser.url(unsecuredurl)
        }
    },

    afterEach: function (browser) {
        if (browser.currentTest.results.errors > 0 || browser.currentTest.results.failed > 0) { browser.end() }
    },

  'Validate Home Page': function(browser){
      if (browser.globals.newSession) {
          // do stuff to navigate the browser to the appropriate page
      }
      browser.page.clients.common().validateHomePage()
    },

[Dharin-shah]

Based on comments/suggestions from other posts on this subject, I coded a workaround that basically checks for any failures after each test case is run and if a failure is found then the browser session is closed. This allows Nightwatch to capture a screenshot. In order to continue the test, I have a beforeEach that checks to see if a new session was started and executes additional actions as necessary. Here's a snippet of the code ...

    before : function (browser) {
        browser.globals.sessionId = browser.sessionId
        browser.url(unsecuredurl)
    },

    beforeEach: function (browser) {
        browser.globals.newSession = false
        if (browser.sessionId != browser.globals.sessionId) {
            browser.assert.log(true, '*** A new session was started \(you may need to handle navigation to the proper starting location\)')
            browser.globals.sessionId = browser.sessionId
            browser.globals.newSession = true
            browser.url(unsecuredurl)
        }
    },

    afterEach: function (browser) {
        if (browser.currentTest.results.errors > 0 || browser.currentTest.results.failed > 0) { browser.end() }
    },

  'Validate Home Page': function(browser){
      if (browser.globals.newSession) {
          // do stuff to navigate the browser to the appropriate page
      }
      browser.page.clients.common().validateHomePage()
    },

This is a nice work-around. However browser.end() now doesnt take screenshot anymore. one can try

 *
 *  this.demoTest = function (browser) {
 *    browser.saveScreenshot('/path/to/fileName.png');
 *  };
 *

So for the above solution it will be like

afterEach: function (browser) {
        if (browser.currentTest.results.errors > 0 || browser.currentTest.results.failed > 0) { browser.saveScreenshot('<filepath\filename>') }
    },

Had a draft PR, but with v2 things will probably change and hence this might not work https://github.com/nightwatchjs/nightwatch/pull/2657