How to handle failed expectations inside of a goroutine?

trivigy commented 5 years ago

I found some conversations about this topic but could not find any workarounds for this issue. If anyone knows I would greatly appreciate the help. Basically I simply want to fail an entire test from within a goroutine running inside of the test. Is there a way to do that and if so how do I do that?

xordspar0 commented 4 years ago

Maybe there should be documentation or a guide on using testify with goroutines in general.

sneko commented 2 years ago

Hi,

@ernesto-jimenez @boyan-soubachov as major contributors to the library, do you have any advice or good reading to follow? It's a bit dark what should be done?

Also @trivigy @xordspar0 in the meantime, did you find the way to go?

Thanks,

EDIT: an old post has a "solution" but it seems a bit... chatty just for failing a test from a goroutine :/ https://github.com/ipfs/go-ipfs/issues/2043#issuecomment-164136026

brackendawson commented 2 years ago

Using testify assert and require with goroutines is the same as using t.Errorf() and t.FailNow() respectively. assert doesn't document this but require sort of does. t.FailNow() must be called from the goroutine running the test, I think this is the issue the OP is mentioning? t.Errorf() does not have the same restriction.

The Go 1.16 release notes also speak about this because calling t.FailNow() in a goroutine became a vet warning (use of the require package does not trigger this warning), they give the most simple example of how to solve it; just use t.Error(). Translating this to testify we would just say "use assert in goroutines, not require".

That method is good enough™️ because either the test will fail, or if the assertion fails after the test has "finshed" then the assertion will cause a panic. But that's not great, the next fix is to use proper synchronisation and the issue that @sneko linked to has reasonable examples of this.

I think this is more of a Go issue than a testify one, but that's not to say testify shouldn't give helpful advice and examples. I'd consider doing:

Document in the assert package that failing assertions will call t.Errorf().
Document in the testify (module root) package that require is not appropriate in a goroutine but assert is.

Expand one of the assertion examples in the testify package to show an assertion in a goroutine, one of these two maybe:


func TestProbablyPanics(t *testing.T) {
go func() {
    // Use assert in goroutines, not require.
    assert.Equal(t, 1, 2)
}()
}

func TestFails(t *testing.T) { syn := make(chan struct{}) // Allow goroutines to finish before the end of the test so their assertions // are reported without a panic defer func() { <-syn }() go func() { defer close(syn) assert.Equal(t, 1, 2) }() }



What do you think?

Antonboom commented 9 months ago

Covered by https://github.com/Antonboom/testifylint#go-require.

Antonboom commented 9 months ago

stretchr / testify

How to handle failed expectations inside of a goroutine? #772