prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
132 stars 38 forks source link

Reports not generated #404

Closed MadhuQAAutomation closed 4 months ago

MadhuQAAutomation commented 5 months ago

Hi @prashant-ramcharan - Using the courgette test runner, I ran my test suite from the command line using 'mvn verify'. After completion of the test, I got the below error and no reports were generated.

Courgette Version : 6.13.0

What could be the issue?

`─────────────────────────────────────────────────── Courgette Test Statistics
─────────────────────────────────────────────────── Summary: 97% passed, 3% failed Duration: 408 min, 3 sec Run Level: Feature Total: 302 Passed: 294 Failed: 8 ───────────────────────────────────────────────────

[Courgette Runner] There was an unexpected error processing the individual Cucumber report files and Courgette was unable to create any reports for this test run.`

I am running the test through junit and below is my test runner configuration.

Test Runner:

@RunWith(Courgette.class) @CourgetteOptions( threads = 1, disableHtmlReport = HtmlReport.COURGETTE_HTML, reportTargetDir = "target", runLevel = CourgetteRunLevel.FEATURE, testOutput = CourgetteTestOutput.CONSOLE, classPath = {"target/libs/*", "target/classes/"}, persistParallelCucumberJsonReports = true, cucumberOptions = @CucumberOptions( snippets = SnippetType.CAMELCASE, features = "src/test/resources/features/policies", glue = {"com.automation.steps"}, plugin = { "pretty", "json:src/test/json/cucumber.json", "html:reports/html-report/automation-report.html", "junit:reports/xml/cucumber.xml" } )) public class TestRunner { }

prashant-ramcharan commented 5 months ago

Hi @MadhuQAAutomation

Courgette combines the reports at the end of the test run and if one of the reports are corrupt / missing the library would be unable to merge all reports into a single report at the end of the run.

As you ran 302 tests this means there were 302 reports stored in memory waiting to be merged into 1 report at the end of the test run and it's likely that one or more reports were not processable.

Have you seen this happen again on a recent run?

MadhuQAAutomation commented 4 months ago

Hi Prashant - I haven't tried again. I will run the suite again and see if the same issue is occurring again.

Thanks for your help

Madhu

On Tue, Apr 30, 2024, 13:29 Prashant Ramcharan @.***> wrote:

Hi @MadhuQAAutomation https://github.com/MadhuQAAutomation

Courgette combines the reports at the end of the test run and if one of the reports are corrupt / missing the library would be unable to merge all reports into a single report at the end of the run.

As you ran 302 tests this means there were 302 reports stored in memory waiting to be merged into 1 report at the end of the test run and it's likely that one or more reports were not processable.

Have you seen this happen again on a recent run?

— Reply to this email directly, view it on GitHub https://github.com/prashant-ramcharan/courgette-jvm/issues/404#issuecomment-2084641072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWIEGE5V5U3KN7EHNHRISYDY75FOVAVCNFSM6AAAAABGCCOYBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBUGY2DCMBXGI . You are receiving this because you were mentioned.Message ID: @.***>

MadhuQAAutomation commented 3 months ago

@prashant-ramcharan - I reran the test suite and got the same error again and the HTML Report is not generated.

image

prashant-ramcharan commented 3 months ago

How many tests did you run by the way?

MadhuQAAutomation commented 3 months ago

@prashant-ramcharan - There were 53 feature files as part of this run.

prashant-ramcharan commented 3 months ago

That doesn't seem a lot to be fair.

I plan on optimising the report processing in a future Courgette release.

MadhuQAAutomation commented 3 months ago

@prashant-ramcharan - Thanks for the update. Also wanted to check if there is any plan to support the generation of courgette reports with multiple JSON files as part of the report optimization.

prashant-ramcharan commented 3 months ago

@MadhuQAAutomation could you elaborate please?

Do you mean generating a Courgette report using multiple Cucumber JSON files outside of a test run?

MadhuQAAutomation commented 3 months ago

@prashant-ramcharan - That's right. If I have 4 different stages in my pipeline and the 4th step is the generation of reports, I want the reports to be generated by reading the JSON files of stage1-3 which are located in the same folder