onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.32k stars 657 forks source link

[Question] Rerun flaky test but only if condition is met #1250

Open cnp-lab opened 1 year ago

cnp-lab commented 1 year ago

Hi, I got a question about flaky tests and rerunning them.

Is it possible to do something similar to what FlakeAttempts does but rerun tests only if given condition/predicate is true? I know it is very specific case but you may have global solution that I can simply apply to my test-suite. E.g. I would like to check if there is specific string in logs and then decide if rerun test when it failed.

onsi commented 1 year ago

Hey there - there isn't formal support for this but you can do something like this:

It("should only rerun if a condition succeeds", FlakeAttempts(5), func() {
    if (CurrentSpecReport().NumAttempts > 1 && !Condition()) {
        Fail("Refusing to rerun spec as condition is not met")
    }
    ...
})

Note that this will only abort in the It - any BeforeEach nodes prior to the It will still run. If that's a problem let me know and I can share some alternatives. But tl;dr - I think you can solve for this without additional Ginkgo APIs since CurrentSpecReport().NumAttempts tells you which attempt this is.