golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
122.91k stars 17.52k forks source link

proposal: testing: consider having timeouts in test binaries be recoverable/handled #44929

Open dprotaso opened 3 years ago

dprotaso commented 3 years ago

The way timeouts are thrown cause the binary to exit immediately https://github.com/golang/go/blob/4d608eb224fe1ba8e8532fcc44f91702a5b17f9f/src/testing/testing.go#L1740-L1744

This circumvents T.Cleanup and any cleanup logic that might have been in TestMain. In order to guarantee setup/teardown occurs it has to be done outside the go test process. Thus either a testing script or the use of go test -exec xprog

The challenge with these workarounds is you lose the simplicity of invoking go test.

Thus my ask is 1) when a timeout occurs run T.Cleanup functions 2) considering surfacing the timeout as a panic but allow the ability to handle that panic

My second point could be accomplished by having the panic be surfaced by M.Run() and could be handled in the following way

func TestMain(m *testing.M) {
   os.Exit(run(m))
}

func run(m *testing.M) (exitCode int) {
  setup()
  defer cleanup()
  exitCode = m.Run()
}
ianlancetaylor commented 3 years ago

Note that the t.Deadline method was introduced specifically so that tests could arrange their own, earlier, timeouts.

dprotaso commented 3 years ago

Yeah - I'm aware of the deadline per test.

My main concern is the setup and teardown that's called out in the TestMain docs

It is sometimes necessary for a test program to do extra setup or teardown before or after testing.