prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
130 stars 38 forks source link

Rerun just failed Scenarios when using CourgetteRunLevel.FEATURE #366

Closed Wolfe1 closed 1 year ago

Wolfe1 commented 1 year ago

When running my tests using CourgetteRunLevel.FEATURE, I wonder if it would be feasible to have just failed Scenarios (not the whole feature) attempt another rerun.

This would be useful for giving more flakey scenarios another chance without having to go through another full feature file run. As far as I can see in the docs this is not possible currently but i wanted to see if I perhaps got this wrong or if you think this would be a feasible future feature?

Thanks!

prashant-ramcharan commented 1 year ago

Hello,

Yes, it is possible just to just rerun the failed scenario when using CourgetteRunLevel.FEATURE.

When using CourgetteRunLevel.SCENARIO its fairly straightforward to merge reports as each scenario will have its own report (including reruns) however using CourgetteRunLevel.FEATURE is a bit more tricky as we need to parse out, replace and rewrite the scenario data within a feature level report.

I will have to do a bit more investigation into the complexity of this.

Wolfe1 commented 1 year ago

@prashant-ramcharan, just some clarity, when you say: "Yes, it is possible just to just rerun the failed scenario when using CourgetteRunLevel.FEATURE."

Do you mean, A. It is possible in the current courgette code implementation or B. It is possible but would require modification to the courgette code to do this?

Thanks.

prashant-ramcharan commented 1 year ago

@Wolfe1 Courgette will require modification to support just re-running the failed scenario when using CourgetteRunLevel.FEATURE

prashant-ramcharan commented 1 year ago

I initially thought that I could get this in the next release but on further testing there was alot of issues with the report generation.

Re-running just the failed scenarios when using CourgetteRunLevel.FEATURE is straightforward however there are alot more complexities around the reporting (json, xml, html, ndjson) which I need to find a solution for.

karthialiasbk commented 10 months ago

I also have the same expectation. Did you fix this issue in any latest version.?