Open OlegKuzovkov opened 1 year ago
@OlegKuzovkov Good day! Could you provide more details please such as IDEA version / libraries versions I tried to reproduce but failed:
I am using:
1. TestNG - 7.7.0
2. allure-testng - 2.23.0
3. allure-java-commons - 2.23.0
4. IDEA - IntelliJ IDEA 2021.1.2 (Community Edition)
Build #IC-211.7442.40, built on June 1, 2021 (yes, I need to update)
Allure TestOps doesn't have a sort by execution order (aka start date).
Describe the bug Our automated testing heavily depends on @BeforeSuite/@BoforeClass/@BeforeMethod methods. When we analyze test results it is crucial for us to see the execution order of tests in the report to quickly understand the sequence of events that cause a failure to happen. When we execute test cases from IDEA, we see test results for classes/beforeMethods/testMethods ordered by the execution:
In Allure the same report will look completely different in terms of execution order:
Even with this simple example this looks bad, but in our case the number of test cases is much bigger that makes our life very complicated when in comes to test result analysis.
To Reproduce Steps to reproduce the behavior:
import lombok.extern.log4j.Log4j2; import org.testng.annotations.*;
import static io.qameta.allure.Allure.step;
@Log4j2 public class TestClass { @BeforeSuite public void beforeSuite() throws Exception { log.info("before suite"); } @BeforeClass public void beforeClass() throws Exception { log.info("before class"); }
}