allure-framework / allure-java

Allure integrations for Java test frameworks
Apache License 2.0
342 stars 219 forks source link

Support AssertJ soft assertions (#19) #988

Open Achitheus opened 6 months ago

Achitheus commented 6 months ago

Context

Step tree

The first change is about processing step tree. If step fails its status should be escalated to all parent steps. (There is another way with substeps check for presence of at least one status differ from PASSED, but I preffered the mentioned above approach as more efficacious). This is achieved due to the following alghorythm:

final Status currentStepStatus = getStepStatus();
        if (currentStepStatus == null) {
            getLifecycle().updateStep(s -> s.setStatus(Status.PASSED));
            getLifecycle().stopStep();
        } else {
            getLifecycle().stopStep();
            if (getLifecycle().getCurrentStep().isPresent()) {
                getLifecycle().updateStep(outerStep -> outerStep.setStatus(currentStepStatus));
            }
        }

Ok, from now on any single failed step will be transformed to the real failed step branch. But do we have firm confidence that on each soft assertion error there is at least one failed step at the end of the step chain? Certainly we dont, I even found a case that confirms this.

The first failed step

To be sure in existence of at least one failed step which can trigger failed status propagating to all outer steps I added advice for setting callback via DefaultAssertionErrorCollector.setAfterAssertionErrorCollected() for each created SoftAssertions object. But there is a problem, because this method is absolutely ordinary setter. It means that its any following call will inevitably overwrite callback that was set before. Fortunately someone has already made corresponding pull request :)

Additionaly

Added pointcut that filters out nested step duplicates created by generated .class files method calls.

Few words about tests

Extraordinary approach is used to avoid a lot of code duplicates and at the same time to cover all possible ways to create soft assertions objects.

Thank you for your attention!

Checklist

CLAassistant commented 6 months ago

CLA assistant check
All committers have signed the CLA.