damianszczepanik / cucumber-reporting

HTML reports for Cucumber
GNU Lesser General Public License v2.1
546 stars 402 forks source link

Total Duration displayed in the report seems to be incorrect when the feature files are executed in parallel #1071

Open MadhuQAAutomation opened 2 years ago

MadhuQAAutomation commented 2 years ago

Hi - The duration value displayed in the cucumber report seems to be incorrect when we feature files were executed in parallel. The time taken for the complete set of feature files to get executed is ~ 39 minutes. But in the report, it shows 3:9:52.041

But this is the actual value :

Summary: 94% passed, 6% failed Duration: 39 min, 9 sec Run Level: Feature Total: 54 Passed: 51 Failed: 3

Same was observed while running the test from JENKINS as well.

File reportOutputDirectory=new File("reports/cucumber/html-reports"); List<String> jsonFiles=new ArrayList<>(); jsonFiles.add("reports/cucumber/json/cucumber.json"); String projectName="cucumber-test-automation"; Configuration configuration=new Configuration(reportOutputDirectory,projectName); //configuration.addPresentationModes(PresentationMode.RUN_WITH_JENKINS); // do not make scenario failed when step has status SKIPPED configuration.setNotFailingStatuses(Collections.singleton(Status.PENDING)); configuration.setNotFailingStatuses(Collections.singleton(Status.UNDEFINED)); configuration.setNotFailingStatuses(Collections.singleton(Status.SKIPPED)); configuration.setSortingMethod(SortingMethod.NATURAL); /*configuration.setTrendsStatsFile(new File("target/trends/trends.json")); configuration.setTrends(new File("target/cucumber/trends/trends.json"),5);*/ configuration.addReducingMethod(ReducingMethod.HIDE_EMPTY_HOOKS); configuration.addReducingMethod(ReducingMethod.MERGE_FEATURES_BY_ID); configuration.addPresentationModes(PresentationMode.PARALLEL_TESTING); configuration.addPresentationModes(PresentationMode.EXPAND_ALL_STEPS); // addidtional metadata presented on main page configuration.addClassifications("Platform","Windows"); configuration.addClassifications("Java","11.0.15"); configuration.addClassifications("Cucumber","7.3.4"); ReportBuilder reportBuilder=new ReportBuilder(jsonFiles,configuration); Reportable result=reportBuilder.generateReports();

image

image

damianszczepanik commented 2 years ago

Report takes all duration and sum them together to have total values. It has no information if the report was generated at once or in parallel.

gerardosanchezqa commented 2 years ago

Hello there, @damianszczepanik !

First of all thanks for working on this reporting tool, it's been super useful in our project.

Question about your last reply: We're running into the same issue as the original poster. Is there a way to add information to cucumber.json to let the report know that it was run in parallel? We tried with PresentationMode.PARALLEL_TESTING but didn't see the duration being adjusted.

damianszczepanik commented 2 years ago

I don't know answer for that question. Probably depends on the framework you use