🐞: Broken/Skipped testStage in the middle of scenario is not failing test case - overall score is PASSED

[MSlaski]

What happened?

Java Test Project with Serenity BDD 4.1.20 (Cucumber 7, Selenium 4, Appium 2, Rest-Assured 5) Some tests are marked with status PASSED even if in the scenario some testStage will be BROKEN or SKIPPED For some reason explicit call of fail() method (doesn't care if Im using JUnit or AssertJ) which is throwing AssertionError is not stopping the tests. On next step test is failing due to timeout of WebElement visibility.

Please check: 7 Test Cases marked as PASSED where the last one really failed

Timeout appeared on 5th Step where rest of the steps were marked as PASSED BUT without details about the sub steps etc

Steps details of working test case:

And the testStages of 5th step:

I didn't hide two methods names, cuz they are the same (called on different classes but both annotated with @Step) - perhaps its worth to mention that

I tried to reproduce the issue on smaller project but without a luck, unfortunately can't provide a test code of my project

What Allure Integration are you using?

allure-awaitility, allure-cucumber7-jvm, allure-junit4, allure-rest-assured

What version of Allure Integration you are using?

2.28.0

What version of Allure Report you are using?

2.28.0

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[MSlaski]

I also made some tests and even assertion like this: assertThat(true).isFalse(); Is not failing the whole test case. Only the particular stage and some stages above. But there is a stage in the middle of particular gherkin step which is passed - dont know why

[MSlaski]

perhaps not an issue with Allure - Cucumber is returning status PASSED for such step

[baev]

@MSlaski Allure alwaysrespectst the status returned by the test framework, so if cucumber is returning the passed, it should be passed in the report (so all works as expected on our side)