prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
133 stars 39 forks source link

Total scenario count is wrong when counted using cucumber hooks #288

Closed BugBuster123 closed 3 years ago

BugBuster123 commented 3 years ago

Hi @prashant-ramcharan,

  1. In my project after executing all the scenarios I need to send an email with total number of scenarios passed, failed, skipped etc apart from the already available html reports. For that I used scenario.getStatus().toString() in after hook of cucumber and stored the count in a static variable, so that after executing each scenario the count will increment. This was working fine for me before using Courgette. But after using Courgette the count is always zero. Am I missing anything?

  2. Does Courgette provide any reports other than html format? something like a pdf format?

prashant-ramcharan commented 3 years ago

Hi @BugBuster123

  1. Courgette runs each test in its own JVM process and static variables are not shared across JVM processes. Static variables will only be available in the context of each test. It will not be shared across all tests.

  2. Currently, Courgette only generates a html report however I can see benefit in creating a PDF report as well. I will consider adding this as a new feature.

A possible option would be to use a Courgette Hook to parse and email the results once all tests are complete.

An example would be to parse the Cucumber xml report to obtain the test statistics (passed, failed, skipped)

JUnit Courgette Runner

@CourgetteOptions(
        ...
        cucumberOptions = @CucumberOptions(
                ...
                plugin = {
                        "junit:build/cucumber-report/cucumber.xml"
                 }
        ))
public class CourgetteRunner {

    @CourgetteAfterAll
    public static void emailTestResults() throws ParserConfigurationException, IOException, SAXException {

         // read the Cucumber xml report
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder = factory.newDocumentBuilder();
        File file = new File("build/cucumber-report/cucumber.xml");
        Document doc = builder.parse(file);

        // parse the results
        int total = Integer.parseInt(doc.getDocumentElement().getAttribute("tests"));
        int failed = Integer.parseInt(doc.getDocumentElement().getAttribute("failures"));
        int skipped = Integer.parseInt(doc.getDocumentElement().getAttribute("skipped"));
        int passed = total - failed - skipped;

        // print out to the console or send via email
        System.out.println("Total tests: " + total);
        System.out.println("Total passed: " + passed);
        System.out.println("Total failed: " + failed);
        System.out.println("Total skipped: " + skipped);
    }
}
BugBuster123 commented 3 years ago

That workaround solved the issue. Thanks for the quick update @prashant-ramcharan .