[Enhancement] Adding screenshots to the xml report (PR ready)

[jpiggg]

Problem Assert does not contain a screenshots field, which is required to insert a screenshot into the xml report, generated by junit.

Description Now I have no way to add screenshots to the xml report. The fact is that - junit has such functionality that is implemented here: https://github.com/nightwatchjs/nightwatch/blob/c6ad58e72dc37e7df9614b78452d11c31e75e8ee/lib/reporter/reporters/junit.xml.ejs#L14 As we can see, if the assert contains a screenshots field, then in the end we will generate the following code:

<system-out>[[ATTACHMENT|/some/path/to/screenshot]]</system-out>

But, the assert object does not contain field screenshots into result: https://github.com/nightwatchjs/nightwatch/blob/c6ad58e72dc37e7df9614b78452d11c31e75e8ee/lib/assertion/assertion.js#L86-L93

I tried to expand this object locally and the screenshots got into the report as I expect above

One way of solution this problem:

1. // nightwatch/lib/assertion/assertion.js
  getAssertResult() {
    return {
      message : this.message,
      stackTrace : this.passed ? '' : this.error.stack,
      fullMsg : this.message,
      failure : this.failureMessage !== '' ? this.failureMessage : false,
      ...this.customFields
    };
  }

2. Modify the assert class so that it can accept custom data
3. Modify Assertion Runner

After that, I can write a custom assert that will receive data with screenshots, from where they will be added to the report.

If this solution is ok, then I can open a PR

[jpiggg]

@beatfactor ping

[jpiggg]

I created PR https://github.com/nightwatchjs/nightwatch/pull/2345 and I use the fork version with my changes and it works good.

[jpiggg]

How I use it

// global.d.ts
import * as EventEmitter from 'events';
import { ICustomAssert } from './lib/customAssert';
import { NightwatchAssertion } from 'nightwatch';

export interface ICustomAssertion<T, U, J = {}> extends NightwatchAssertion<T, U> {
    data: J
}

export interface INightwatchCustomAsserts {
    customOk: ICustomOk
}

declare module 'nightwatch' {
    export interface NightwatchCustomAssertions extends ICustomAssertion<boolean, any, {[key: string]: string[]}>, EventEmitter {}
    export interface NightwatchAPI extends NightwatchCustomAssertions {}
}

// customAssert.ts
import { NightwatchAssertions } from 'nightwatch';
import { ArgumentsType } from '../utils/args';

export type IData = {[key: string]: any};

export type ICustomAssert = (val: boolean, message: string, data: IData) => void;

type Args = ArgumentsType<ICustomAssert>
/**
 * Abstract assertion class that will subclass all defined assertions
 *
 * All assertions must implement the following api:
 *
 * - @param {T|function} expected
 * - @param {string} message
 * - @param {function} pass
 * - @param {function} value
 * - @param {function} command
 * - @param {function} - Optional failure
 */

exports.assertion = function(this: NightwatchAssertions, ...[val, message, data]: Args) {
  this.expected = val;
  this.message = message;

  this.data = {...data};

  this.pass = function(value) {};

  this.value = function(result: any) {};

  this.command = function(callback) {
    callback(/* */);

    this.emit('complete')

    return this;
  }
}

And this.data avialable at my custom reporter

[jpiggg]

@beatfactor Hey! Why are you not answer for my issue? It really actual and I wait so long

[beatfactor]

@Dolinskaya Sorry for the delay, I will get to it soon. But this looks more like bug, doesn't it? The screenshots functionality is there, but it's not working properly.

[jpiggg]

@beatfactor Not quite. As far as I know, the screenshots functionality is only on error assertions. But I want, that it will be always into asserts. And also, I want to add custom logic for asserts. For example, for our custom reporter I use 3 screenshots: source screenshot, reference screenshot and diff. I'm more comfortable, when screenshots are array of objects with title and path to screenshot. Now screenshots on error are array with paths to screenshots.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.

[pawlakmaly]

Question,any update on this issue ? It's been quite a while and the xml reports are still missing the screenshots. Or I don't know how to turn this on.

[beatfactor]

@pawlakmaly we'll address this soon.

[pawlakmaly]

@beatfactor Is there any chance that this change/fix will be added to upcoming version ? It would make the test reports on CI/CD systems like azure to look great..

[KarthikKallu-ISelect]

any updates @beatfactor on this , could u please add this enhancement early , Thanks In Advance for ur time.

[KarthikKallu-ISelect]

@beatfactor could u please add that one enhancement of adding screenshots to html/any other reporting type /please suggest how to add screenshots to html/xml report

[pawlakmaly]

@KarthikKallu-ISelect , try this code to log screenshot to report:

this.client.__reporter.testResults.logScreenshotFile(fullScreenshotPath);

[KarthikKallu-ISelect]

Hi @pawlakmaly Thanks for your time in answering it ,I will check and let u know on it

[KarthikKallu-ISelect]

Hi @pawlakmaly May i know the correct usage , could u please let me know on how to use it,I do write tests something like below in typescript much appreciated ur time. reporting tool used :- https://testersdock.com/nightwatch-js-html-report/ I do write tests like below

module.exports={ 'verify contact number 19 29 40 displayed in home page': function (browser: NightwatchBrowser) { const home = new HomePage(); home.verifyContactNumber(browser) .getText(home.elements.Contact, (result) => { browser.assert.equal(result.value, '19 29 40') }) .end(); },

beforeEach: function (browser: NightwatchBrowser) {
    browser
        .resizeWindow(1400, 900)
        .url(browser.launch_url)
    browser.globals.abortOnAssertionFailure=false
},

afterEach: function (browser: NightwatchBrowser) {
    // browser.end()
    const util = new Util();
    let path='./screenshots/' + browser.currentTest.name + "__" + util.random(8) + '.png'
    browser.saveScreenshot(path)  
  browser .__reporter.testResults.logScreenshotFile(path);
},

};

[pawlakmaly]

@KarthikKallu-ISelect I use nunit3 reported and the code I've passed to you is in my custom command. I'm not sure if this will work in afterEach hooks. Try use it inside your test case.

In my case I use it more or less in that way:

"my simple test case": async (browser) => {
await browser..verify.cssClassPresent("some parameters");
await browser.makeScreenshotOnAssertionError() //this is my custom command which makes the screen with name of current test case and attach the screenshot using method I've mention above.
}