Open brandur opened 1 year ago
I was curious too, so this is for future Googlers. The README says that this tool was inspired by a blog post: Go 言語でのテストの並列化 〜t.Parallel()メソッドを理解する〜. I used Google Translate to get the following translation of the last section, which is helpfully labeled "defer
statement and t.Cleanup()
method." (Two notes: (1) the chunk I translated refers to code from earlier in the blog post, but I didn't want to translate and copy everything. I think it's clear enough, but YMMV. (2) I tweaked the translation in a few cases where I thought it was gibberish. Refer back to the original article (and retranslate) as needed.)
When doing post-processing at the end of the test, you need to be careful whether you use a
defer
statement or at.Cleanup()
method. As for the top-level test functions, the basics are:
- If the top-level test function does not contain a subtest function with a
t.Run()
method, then either adefer
statement or at.Cleanup()
method may be used to perform the post-processing.- A
defer
statement or at.Cleanup()
method (both work) if a top-level test function contains subtest functions with at.Run()
method and none of its subtest functions call at.Parallel()
method. You can perform the post-processing either way.- If a top-level test function contains subtest functions with
t.Run()
methods and at least one of those subtest functions calls thet.Parallel()
method, then perform post-processing with thet.Cleanup()
method.A
defer
statement is called when the function containing the statement returns. Review the example execution of the preceding code. TheTest_Func1
function returns before theFunc1_Sub1
andFunc1_Sub2
subtest functions have finished (Action 2). Therefore, the function with the delay specified in thedefer
statement included in theTest_Func1
function is called before the subtest functions ofFunc1_Sub1
andFunc1_Sub2
resume processing after being paused (in the above execution result, note the position ofTest_Func1
returned).For example, even if the top-level test function delays a post-processing call, such as deleting a record in a table created by the subtest function, with a
defer
statement, if the subtest function calls thet.Parallel()
method, the post-processing function specified in thedefer
statement will be called prior to the execution of the subtest function. In such cases, write the cleanup using thet.Cleanup()
method instead of the defer statement.The specification of the
t.Cleanup()
method is as follows.
func (c *T) Cleanup(f func())
Cleanup registers a function to be called when the test and all its subtests complete. Cleanup functions will be called in last added, first called order.The function registered with the
t.Cleanup()
method is said to be called when all subtests are finished.Then, what happens to the post-processing in the subtest function described by the
t.Run()
method is as follows (basically the same as the above three items, but the subject is different).
- If the subtest function does not contain further subsubtest functions with nested
t.Run()
methods, then either thedefer
statement or thet.Cleanup()
method may be used to describe the post-processing.- A
defer
statement ort.Cleanup()
if a subtest function contains further subsubtest functions with nestedt.Run()
methods and none of the subsubtest functions call thet.Parallel()
method. You can write post-processing in either method.If the sub-test function contains further sub-sub-test functions with nested
t.Run()
methods and at least one sub-sub-test function calls thet.Parallel()
method, then perform post-processing with thet.Cleanup()
method.If you are too lazy to memorize the above six items about top-level test functions and sub-test functions, some projects say, "In test code that uses the
t.Parallel()
method, the post-processing ist.Cleanup( )
."
Ah, thanks for tracking that down @telemachus. I'm going to add a couple snippets from the blog post here as well for easier reference.
The implementation of Func1_Sub1
and Func1_Sub2
referenced above:
func Test_Func1(t *testing.T) {
defer trace("Test_Func1")()
t.Run("Func1_Sub1", func(t *testing.T) {
defer trace("Func1_Sub1")()
t.Parallel()
// ...
})
t.Run("Func1_Sub2", func(t *testing.T) {
defer trace("Func1_Sub2")()
t.Parallel()
// ...
})
// ...
}
And a sample run where you can see the top-level Test_Func1
defer running before the subtests finish:
=== RUN Test_Func1
Test_Func1 entered
=== RUN Test_Func1/Func1_Sub1
Func1_Sub1 entered <- Func1_Sub1が開始
=== PAUSE Test_Func1/Func1_Sub1 <- Func1_Sub1が一時停止
=== RUN Test_Func1/Func1_Sub2
Func1_Sub2 entered <- Func1_Sub2が開始
=== PAUSE Test_Func1/Func1_Sub2 <- Func1_Sub2が一時停止
Test_Func1 returned <- Test_Func1の呼び出し戻り(*)
=== CONT Test_Func1/Func1_Sub1 <- Func1_Sub1が再開
Func1_Sub1 returned <- Func1_Sub1が完了
=== CONT Test_Func1/Func1_Sub2 <- Func1_Sub2が再開
Func1_Sub2 returned <- Func1_Sub2が完了
--- PASS: Test_Func1 (0.00s) <- Test_Func1の結果表示
--- PASS: Test_Func1/Func1_Sub1 (0.00s)
--- PASS: Test_Func1/Func1_Sub2 (0.00s)
=== RUN Test_Func2 <- ここまでTest_Func2は実行されない
Test_Func2 entered
=== PAUSE Test_Func2
=== RUN Test_Func3
Test_Func3 entered
Test_Func3 returned
--- PASS: Test_Func3 (0.00s)
=== RUN Test_Func4
Test_Func4 entered
=== PAUSE Test_Func4
=== RUN Test_Func5
Test_Func5 entered
Test_Func5 returned
--- PASS: Test_Func5 (0.00s)
=== CONT Test_Func2
Test_Func2 returned
=== CONT Test_Func4
Test_Func4 returned
--- PASS: Test_Func4 (0.00s)
--- PASS: Test_Func2 (0.00s)
PASS
If I had to summarize the problem: use of t.Parallel()
in tests may cause them to be paused while the runner is working on other things. While paused, a parent test function may return, which would execute any defer
statements it had registered. Use of t.Cleanup
is the workaround because although it behaves similarly to a defer
, it's baked into the test framework so that it can guarantee the right thing will happen.
tparallel
will require the use oft.Cleanup
instead ofdefer
in cases where a top-level test case has subtests, and that top-level test case ist.Parallel
. From the README:I was bringing
t.Parallel
into a project and had to make some changes related to this, but was having a hard time explaining to my colleague why exactly those changes are necessary.Could you elaborate a little more on why exactly
t.Cleanup
needs to be used instead ofdefer
in some cases? I can help expand the README to explain it if you'd like (after I understand it myself that is).