teardown failure hides panic

[rogpeppe]

Consider this code:

package main

import (
    "testing"

    gc "gopkg.in/check.v1"
)

type suite struct {
    closed bool
}

var _ = gc.Suite(&suite{})

func TestP(t *testing.T) {
    gc.TestingT(t)
}

func (s *suite) SetUpTest(c *gc.C) {
    s.closed = false
}

func (s *suite) TestSomething(c *gc.C) {
    panic("this panic does not become visible")
    s.closed = true
}

func (s *suite) TearDownTest(c *gc.C) {
    if !s.closed {
        c.Fatalf("not closed")
    }
}

This prints:

----------------------------------------------------------------------
FAIL: tst_test.go:28: suite.TearDownTest

tst_test.go:30:
    c.Fatalf("not closed")
... Error: not closed

----------------------------------------------------------------------
PANIC: tst_test.go:23: suite.TestSomething

... Panic: Fixture has panicked (see related PANIC)
OOPS: 0 passed, 1 FAILED, 1 MISSED
--- FAIL: TestP (0.00s)
FAIL
FAIL    command-line-arguments  0.002s

The panic has been hidden because the teardown failed. This isn't that unusual. For example in the Juju test suite, there's a check in TearDownTest that all the mongo sessions are closed - if test doesn't proceed to completion because of a panic and as a result does not close a session, the failure is confusing as it's not clear that there has been a panic at all.

In fact the output is identical to the output that happens when the TearDownTest has an error after the actual test has passed successfully.

[rogpeppe]

A simpler example:

package main

import (
    gc "gopkg.in/check.v1"
    "testing"
)

type suite struct{}

var _ = gc.Suite(&suite{})

func TestPackage(t *testing.T) {
    gc.TestingT(t)
}

func (s *suite) TearDownTest(c *gc.C) {
    c.Error("TearDownTest failed")
}

func (s *suite) TestSomething(c *gc.C) {
    panic("something went horribly wrong")
}

[Blackmage89]

Hi, maybe is not strictly the same bug, but also the Failed() method does not report correctly the result of the tests if invoked in the teardown method

package main

import (
    gc "gopkg.in/check.v1"
    "testing"
    "fmt"
)

type suite struct{}

var _ = gc.Suite(&suite{})

func TestPackage(t *testing.T) {
    gc.TestingT(t)
}

func (s *suite) TearDownTest(c *gc.C) {
    fmt.Println("Failed() In TearDown of Test: ", c.Failed())
}

func (s *suite) TestSomethingAssert(c *gc.C) {
    defer func(){
        fmt.Println("Failed() In Defer of Test: ", c.Failed())
    }()
    c.Assert(1,gc.Equals,0)
}

Which outputs:

> go test example_test.go
Failed() In Defer of Test:  true
Failed() In TearDown of Test:  false

----------------------------------------------------------------------
FAIL: example_test.go:21: suite.TestSomethingAssert

example_test.go:25:
    c.Assert(1, gc.Equals, 0)
... obtained int = 1
... expected int = 0

OOPS: 0 passed, 1 FAILED
--- FAIL: TestPackage (0.00s)
FAIL
FAIL    command-line-arguments  0.273s