running single(or multiple tests one at a time) nightwatch test to generate allure report hangs after the test is finished.

[dheeraj7771]

i am using this adaptor to generate allure report, it works fine for builds which runs more than one test in parallel, how ever when the test run one at a time, the build/test hangs at the finishing of last test. the issue doesn't seems to be with nightwatch itself as the builds/tests with single test execution is running from long time now and never hanged. i notice the hanging issue when i run a single test locally as well.

basically when selenium starts in parallel mode the report doesn't hang but when selenium starts a single process. allure report hangs.

is there any setting to let allure know that the single test is running

[dheeraj7771]

by adding , "test_workers": { "enabled": true, "workers": 1 } in nightwatch.json for each environment(that needs to runs single test at a time), resolves the problem. as this helps in starting the selenium process in parallel mode when used with --tag "test case unique tag". however the problem still persists when the test is run using --test "test_name.js" as the "--test" command of nightwatch does not start the selenium in parallel mode.

[sharadJay]

do you still observe this issue?

[vinayakshenoy91]

I have a query.I am new to node. I have the nightwatch xml created in reports directory. After adding reporter in globals.js as mentioned , what is the next step for execution? Do we need to type any command?

[sharadJay]

you can use this commandline to generate allure report locally https://www.npmjs.com/package/allure-commandline on terminal else if you are using tools like jenkins or teamcity there are plugins to generate reports available. Refer this https://docs.qameta.io/allure/#_reporting

[vinayakshenoy91]

I assume this command-line is used once the allure compatible reports are generated. I have the report generated by nightwatch in the below format. I assume your adapter converts this to allure compatible format and generates XML which can then be opened/generated using the allure command-line mentioned by you above.

Nightwatch generate XML:

Command to run and generate the above xml/nightwatch cases: "scripts": { "nightwatch": "nightwatch -c nightwatch.conf.js -e chrome" },

Folder stucture:

Iam not able to understand how do we generate allure compatible xmls from the xml generated by nightwatch.

[sharadJay]

We dont convert junit reports from nightwatch but create allure reports directly using custom reporter https://github.com/sharadJay/nightwatch-allure-adapter#using-reporter-in-nightwatch

[mahajanavi]

As an workaround, you can update reporter.js file like below:

File: nightwatch-allure-adapter\reporter.js

var Allure = require("allure-js-commons"); var allureReporter = new Allure(); var Runtime = require("allure-js-commons/runtime"); var fs = require("fs"); var path = require("path"); var cp = require("comment-parser"); var _ = require('lodash'); var runtimeAllure = new Runtime(allureReporter); var find = require("find");

var self = module.exports = { write: function (results, done, directoryPath) { allureReporter.setOptions(" -o reports/allure-report" || {}); for (var currentModule in results.modules) { module = results.modules[currentModule]; var currentTest = { failures: self.parse(module.failures), errors: self.parse(module.errors), skipped: self.parse(module.skipped.length), tests: self.parse(module.tests), isFailure: false, isSkipped: false, suiteName: module.group, testName: currentModule, testSteps: [], errorMessage: "", startTimestamp: self.parseDate(module.timestamp), endTimestamp: self.parseDate(module.timestamp), tags: {} }

        if (typeof directoryPath !== 'undefined') {
            currentTest.tags = self.parseFileForTags(directoryPath + "/tests/" + currentModule + ".js");
        }

        if (currentTest.skipped === currentTest.tests) {
            currentTest.isSkipped = true;
        } else if (currentTest.failures > 0 || currentTest.errors > 0) {
            currentTest.isFailure = true;
        }
        var testPath = currentTest.testName.split("/");

        if (currentTest.suiteName === undefined) {
            currentTest.suiteName = testPath[testPath.length - 2];
        }
        if (currentTest.suiteName === "") {
            currentTest.suiteName = "Default Suite"
        }
        if (results.hasOwnProperty("environment")) {
            currentTest.suiteName = currentTest.suiteName + "-" + results.environment;
        }
        if (testPath.length > 1) {
            currentTest.testName = testPath[testPath.length - 1];
        }

        allureReporter.startSuite(currentTest.suiteName, currentTest.startTimestamp);
        allureReporter.startCase(currentTest.testName, currentTest.startTimestamp);
        //TODO considering good number of properties switch should be used
        if (currentTest.tags.hasOwnProperty("testcaseId")) {
            runtimeAllure.addLabel("testId", currentTest.tags["testcaseId"])
        }
        if (currentTest.tags.hasOwnProperty("description")) {
            runtimeAllure.description(currentTest.tags.description);
        }
        allureReporter.addAttachment("Reported Result", JSON.stringify(results), "application/json");
        var previousStepTimestamp = currentTest.startTimestamp;

        for (var completedStep in module.completed) {
            var currentStep = module.completed[completedStep];

            var curCompletedStep = {
                failures: self.parse(currentStep.failed),
                errors: self.parse(currentStep.errors),
                skipped: self.parse(currentStep.skipped),
                passed: self.parse(currentStep.passed),
                startTimestamp: previousStepTimestamp,
                endTimestamp: previousStepTimestamp + (self.parseFloat(currentStep.time) * 1000),
                totalTime: self.parseFloat(currentStep.time) * 1000
            }
            currentTest.endTimestamp = currentTest.endTimestamp + curCompletedStep.totalTime;
            previousStepTimestamp = curCompletedStep.endTimestamp;
            allureReporter.startStep(completedStep, curCompletedStep.startTimestamp);
            for (assertion in  currentStep.assertions) {
                allureReporter.startStep(currentStep.assertions[assertion].message, curCompletedStep.startTimestamp);
                allureReporter.endStep("passed", curCompletedStep.endTimestamp);
            }
            if (curCompletedStep.failures > 0 || curCompletedStep.errors > 0) {
                allureReporter.endStep("failed", curCompletedStep.endTimestamp);
                for (var assertion in currentStep.assertions) {
                    var currentAssertion = currentStep.assertions[assertion];
                    if (currentAssertion.failure != false) {
                        var errorMessage = {
                            failure: currentAssertion.failure,
                            message: currentAssertion.message,
                            stacktrace: currentAssertion.stacktrace
                        }
                        currentTest.errorMessage = {
                            message: errorMessage.failure + errorMessage.message,
                            stack: errorMessage.message + "\n" + errorMessage.failure + "\n" + errorMessage.stacktrace
                        }
                    }
                }
            } else {
                allureReporter.endStep("passed", curCompletedStep.endTimestamp);
            }

        }

        for (var skippedStep in module.skipped) {
            allureReporter.startStep(module.skipped[skippedStep], currentTest.endTimestamp);
            allureReporter.endStep("skipped", currentTest.endTimestamp);
        }

        if (currentTest.isFailure) {
            if (typeof directoryPath !== 'undefined') {
                find.file(/\.png$/, directoryPath + "/screenshots/" + results.environment + "/" + currentModule, function (files) {
                    files.forEach(function (file) {
                        fs.readFile(file, function (err, data) {
                            allureReporter.addAttachment("screenshots", data, "image/png");
                            allureReporter.endCase("failed", currentTest.errorMessage, currentTest.endTimestamp);
                            allureReporter.endSuite(currentTest.endTimestamp);
                        });
                    });
                });
            } else {
                allureReporter.endCase("failed", currentTest.errorMessage, currentTest.endTimestamp);
                allureReporter.endSuite(currentTest.endTimestamp);
            }

        } else if (currentTest.isSkipped) {
            allureReporter.endCase("skipped", "No Steps Performed", currentTest.endTimestamp);
            allureReporter.endSuite(currentTest.endTimestamp);
        }

        else {
            allureReporter.endCase("passed", "", currentTest.endTimestamp);
            allureReporter.endSuite(currentTest.endTimestamp);
        }

    }
    /**
     * Below Code added to terminate report process gracefully
     *
     * After all the tests are run, evaluate if there were errors and exit appropriately.
     *
     * If there were failures or errors, exit 1, else exit 0.
     *
     * @param results
     */
    if ((typeof(results.failed) === 'undefined' || results.failed === 0) &&
        (typeof(results.error) === 'undefined' || results.error === 0)) {
        process.exit(0);
    } else {
        process.exit(1);
    }
    done();
},
parse: function (str) {
    return _.isNaN(str) ? 0 : parseInt(str, 10);
},
parseFloat: function (str) {
    return _.isNaN(str) ? 0 : parseFloat(str);
},
parseDate: function (str) {
    return Date.parse(str);
},
//FIXME file paths are incorrect, hence can not use this
parseFileForTags: function (testfilePath) {
    var opts = {
        parsers: [
            cp.PARSERS.parse_tag,
            cp.PARSERS.parse_description,
        ]
    };

    var file = fs.readFileSync(testfilePath, 'utf-8');
    var parsedInformation = cp(file, opts);
    var tcTags = {};
    if (parsedInformation.length > 0) {
        tcTags.description = parsedInformation[0].description;
        var tagsInTest = parsedInformation[0].tags;
        for (var tag in tagsInTest) {
            currentTag = tagsInTest[tag];
            switch (currentTag.tag) {
                case "testcaseid":
                    tcTags.testcaseId = currentTag.description;
                    break;
                case "type":
                    tcTags.type = currentTag.description;
                    break;
                case "testtype":
                    tcTags.testType = currentTag.description;
                    break;
            }
        }
    }
    return tcTags;
}

};

[sharadJay]

@mahajanavi I did not get it, what changes did you make in reporter.js?

[mahajanavi]

@sharadJay I have added below lines, they simply terminate process.

if ((typeof(results.failed) === 'undefined' || results.failed === 0) && (typeof(results.error) === 'undefined' || results.error === 0)) { process.exit(0); } else { process.exit(1); }

Ref https://stackoverflow.com/questions/33409800/nightwatch-js-pausing-at-the-end-of-test-suite

[sharadJay]

Thanks for the info, I will try to add this soon

[govind413]

@vinayakshenoy91 you can follow steps from:

https://stackoverflow.com/questions/50783032/how-to-generate-allure-report

1. Add this dependency by running npm install allure-commandline --save-dev
2. Run your tests and generate test result data (ie, after running it will generate allure-results folder).
3. From the same project directory run, allure generate allure-results --clean -o allure-report in the command prompt
4. On successfull execution it will generate one more folder allure-reportin your directory.
5. Open index.html file in FireFox to show the report.

It works nicely.

[govind413]

@mahajanavi Hi,

After replacing reporter.JS with your solution, 1st time it is working fine i.e. able to create xml results as well as exit tests normally but from 2nd time onwards selenium itself is not running. getting message like 'Starting selenium server...' and no actions taken further