Automatic screenshots not taken when 'end_session_on_fail' is false

[hukka]

If 'end_session_on_fail' in 'test_settings' is true, no screenshots are taken when asserts fail. This was a bit surprising, so if that's as designed, then it would be nice to have it in the docs.

[beatfactor]

Can you explain your use case for end_session_on_fail? The screenshot is being called now from the .end() command automatically, regardless of the end_session_on_fail setting.

On Wed, Oct 7, 2015 at 1:27 PM, Tomi Hukkalainen notifications@github.com wrote:

If 'end_session_on_fail' in 'test_settings' is true, no screenshots are taken when asserts fail. This was a bit surprising, so if that's as designed, then it would be nice to have it in the docs.

— Reply to this email directly or view it on GitHub https://github.com/nightwatchjs/nightwatch/issues/673.

[hukka]

I was trying the multiple step example at http://nightwatchjs.org/guide#writing-tests combined with screenshots and making it fail at different points, and without end_session_on_fail it never progresses to the second step, if first has assert failures.

Now I'm splitting the tests into separate files. If this is the designed way to organize the tests, perhaps a short note could be added to the docs on multi step tests?

[beatfactor]

Btw, that setting is true by default so it is strange the screenshots don't work for you. I need to see some code to try to reproduce and os, nightwatch and nodejs versions.

On Wednesday, October 7, 2015, Tomi Hukkalainen notifications@github.com wrote:

I was trying the multiple step example at http://nightwatchjs.org/guide#writing-tests combined with screenshots and making it fail at different points, and without end_session_on_fail it never progresses to the second step, if first has assert failures.

Now I'm splitting the tests into separate files. If this is the designed way to organize the tests, perhaps a short note could be added to the docs on multi step tests?

— Reply to this email directly or view it on GitHub https://github.com/nightwatchjs/nightwatch/issues/673#issuecomment-146192214 .

[ResolverJay]

Just realized I have the same issue. I don't think there's anything to do with "end_session_on_fail" or "skip_testcases_on_fail", since I tried both true and false, none of the combination worked. (tried with 0.8.3 & 0.8.4)

"screenshots" : {
        "enabled" : true,
        "on_failure" : true,
        "on_error" : true,
        "path" : "reports"
},

[beatfactor]

os?

On Wed, Oct 7, 2015 at 9:47 PM, ResolverJay notifications@github.com wrote:

Just realized I have the same issue. I don't think there's anything to do with "end_session_on_fail" or "skip_testcases_on_fail", since I tried both true and false, none of the combination worked. (tried with 0.8.3 & 0.8.4)

"screenshots" : { "enabled" : true, "on_failure" : true, "on_error" : true, "path" : "reports" },

— Reply to this email directly or view it on GitHub https://github.com/nightwatchjs/nightwatch/issues/673#issuecomment-146308224 .

[ResolverJay]

Windows 8.1

[hukka]

Seems I was confused; the problem happens when end_session_on_fail is not the default, i.e. false. A bit more comprehensive testing with skip_test_case_on_fail included:

neither false: will skip tests after failing, will create screenshots only skip_test_case_on_fail is false: will run all tests even after failing, but since the session has ended the browser would need to reload url, will create screenshots only end_session_on_fail is false: will skip tests after failing, will not create screenshots both false: will run all tests even after failing, latter tests work without url command, will not create screenshots

This happens both on assert and expect style tests, not that I expected it to make a difference.

[xtepwxly]

Windows 10 - the same problem. I have this configuration:

  "end_session_on_fail": false,
  "skip_testcases_on_fail" : false,
  "screenshots" : {
    "enabled" : true,
    "on_failure" : true,
    "on_error" : true,
    "path" : "screenshots"
  },

Maybe somebody knows how to avoid this problem? Need to take screenshots when assertion failed or error occured

[mattpardee]

+1 I have "end_session_on_fail": false, and it doesn't save screenshots. Set it to true and it does.

[mattpardee]

@beatfactor it looks like if end_session_on_fail is false then the following happens when this code is run in lib/api/client-commands/end.js:

End.prototype.testFailuresExist = function() {
  return this.client.results.errors > 0 || this.client.results.failed > 0;
};

If I console.log(this.client.results); in there, it shows this:

{ passed: 0,
  failed: 0,
  errors: 0,
  skipped: 0,
  tests: [],
  lastError: 
   { [Testing if the URL...

Interesting that failed is 0 and none of the other numbers are set. However, if I set end_session_on_fail: true then the object is properly populated with the right numbers..

{ passed: 6,
  failed: 1,
  errors: 0,
  skipped: 0,
  tests: 
   [ { message:

I'm probably going to debug into this more, just wanted to provide some results if it might get your mind on the solution path!

[shacall]

+1 I have the same problem

  "screenshots" : {
    "enabled" : true,
    "on_failure" : true,
    "on_error" : true,
    "path" : "screenshots"
  },
  "end_session_on_fail": false,
  "skip_testcases_on_fail" : false,

[corydolphin]

@beatfactor is there any update on this? I'm running into this.

I set end_session_on_fail to false to allow us to grab error console logs from the browser for debugging purposes.

[Rikusor]

+1 I also have this issue, I have following configuration;

 "test_settings": {
    "default": {
      "abortOnAssertionFailure": false,
      "selenium_host": "127.0.0.1",
      "end_session_on_fail": false,
      "skip_testcases_on_fail": false,
      "screenshots" : {
        "enabled" : true,
        "on_failure": true,
        "on_error": true,
        "path" : './reports/e2e/'
      },
      "globals": {
        "waitForConditionTimeout": 5000
      },
      "desiredCapabilities": {
        "browserName": "chrome"
      }
    },
    "chrome": {
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true // turn off to test progressive enhancement
      }
    }
  }

If I remove the "end_session_on_fail": false, or change it to true screenshots are generated correctly.

Any updates on this? Will this be fixed or is it working as designed?

---

UPDATE @beatfactor @

Did some digging, still not sure how everything works and are linked together, but my two cents;

On file /lib/index.js there is method Nightwatch.prototype.terminateSession

Where is;

  if (this.options.end_session_on_fail && this.options.start_session) {
    this.api.end(function() {
      this.finished();
    }.bind(this));

    // FIXME: sometimes the queue is incorrectly restarted when another .end() is
    // scheduled from globalBeforeEach and results into a session command being sent with
    // null as the sessionId
    this.queue.run();
  } else {
    this.finished();
  }

The api.end function is not called ifend_session_on_fail is set to false.

Inside the api.end method is;

if (this.testFailuresExist() && this.shouldTakeScreenshot()) {
      var fileNamePath = Utils.getScreenshotFileName(client.api.currentTest, false, client.options.screenshots.path);
      Logger.info('We have failures in "' + client.api.currentTest.name + '". Taking screenshot...');

      client.api.saveScreenshot(fileNamePath, function(result, err) {
        if (err || result.status !== 0)  {
          Logger.warn('Error saving screenshot...', err || result);
        }
      });
    } 

Could this be the cause for the issue?

[straris]

Still valid on 0.9.19, To be precise: Setting end_session_on_fail to false will result in no screenshots

[amarbir22]

Any update on this? I'm Facing this issue with => abortOnAssertionFailure: false OS High Sierra 10.13.6 Nightwatch 0.9.21

[ClaytonAstrom]

I think I accidentally duplicated this in #1922 , I believe #1925 should fix it?

[stale[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.