Assert: go test output shows subtests that fail as PASS

[Anacardo89]

When running a test with various subtests with t.Run(), set to fail, the test itself fails, but the substests all show as passed in the end, so there's no quick way of checking which subtest failed, apart from checking the line in the editor.

This doesn't happen if using the std library instead of assert

Without t.Parallel() it gets easier to tell which subtest failed, since they fail as they're called, but that's a strenuous limitation.

[Anacardo89]

So i fiddled around with if for a bit and find that it doesn't reproduce if i don't create a new assert and just pass t everytime i call it:

So there's a workaround at least.

I looked around the t.Run and assert implementations and got confused but i think it's related to assert.t not being the same as t that is used in t.Run

so when Fail() calls t.Errorf(), which calls it's own Fail() it's modifying different t's?

But if that was the case then it wouldn't only not set c.failed, i would also not set c.parent with Fail()

As i said, i'm confused, this is confusing, imma go study interfaces now and hope that sheds some light into the matter, at least to me.

[tscales]

You're basically on the right track. The top level testing.T that you pass in to assert.New and the testing.T created when invoking t.Run are different. In your first example, no assertions are made against your subtest so its reports passing. You would basically have to do something like this.

func TestNotNil(t *testing.T) {
    a := assert.New(t)
    t.Run("Test Not Nil Sub", func(t *testing.T) {
        passed := a.NotNil(t)
                if !passed {
                    t.Fail()
                }
    })
}

but as you discovered, you can skip using assert.New all together.

[Anacardo89]

I get it, since i'm declaring a outside t.Run(), even if i use it inside, it will reply to the whole test so i have to manually fail t.Run(). a isn't failing the parent, it still IS the parent and fails itself. Thank you, this insight was really helpful