Support colorized output, indentation in accordance with subtest depth, and one more assertion

[jsf116]

The implementation experience gathered by our team showed that the following features currently implemented by our own would improve the overall maintainability of a module to be tested:

1. If the verbose mode is applied, display all test results including positive ones.
2. Colorize output displaying positive results in green and negative ones in red.
3. Indent test ouput to the right in accordance with subtest depth.
4. Mock (config) file needed for a test by copying of template file to a temporary one and replacing some parts of its content during the copying in accordance with the list of pairs search pattern (RegEx) -> replacement string supplied in form of a map.

Please let me know if these features can be implemented. Thank you in advance!

[tscales]

Colorize output displaying positive results in green and negative ones in red.

Colorization has been discussed in other issues. See https://github.com/stretchr/testify/labels/enhancement%3A%20colored%20output. There are a few PRs that have been proposed.

As for your other three points. Could you please provide examples of output you are seeing and what you would like to see?

[jsf116]

Regarding 1. and 3. Assuming the following test located in the function TestIndentation in lines 10 - 13 of the source file /my_path/my_test.go is executed:

assert.True(t, true, "test on the highest level")
t.Run("subtest visualization", func(t *testing.T) {
    assert.True(t, false, "deeper level test")
})

Then the expected output would be something like

=== RUN   TestIndentation
...
    /my_path/my_test.go:10:                                         <----- in green
    test on the highest level                                       <----- in green
=== RUN   deeper_level_test
...
        /my_path/my_test.go:12:                                     <----- in red
        deeper level test                                           <----- in red

Regarding 4. Assuming the application is using a configuration file config.yamlcontaining the name of some necessary directory in the parameter my_dir and the number of some iterations in the parameter my_num. Assuming furthermore that a temporary directory and a random number from 0 to 100 should be used instead in some test cases. Then tests related to this specific can take the template yaml file called e.g. config_template.yaml, copy this file to the temporary directory created on the fly and replace the value of my_dir with the name of this temporary directory and the value of my_num with a random integer during the copying:

    tempDir := t.TempDir()
    numOfIter := rand.Intn(100)
    replacements := map[string]string{
        `(?m)^my_dir:\s*\S+$`: "my_dir: " + tempDir,
        `(?m)^my_num:\s*\S+$`: "my_num: " + numOfIter,
    }
    mock.PrepareFileContent("config_template.yaml", "config.yaml", replacements)
    // some tests

[tscales]

regarding indentation, subtests results are already displayed (and indented) in verbose mode. I think what you are really asking is to display assertion results in verbose mode.

as subtests work today (unrelated to testify)

func TestMain(t *testing.T) {

    assert.True(t, true, "true should be true")
    t.Run("subtest visualization", func(t *testing.T) {
        assert.True(t, true, "true should be true")
        t.Run("sub-subtest visualization", func(t *testing.T) {
            assert.True(t, true, "true should be true")
        })
    })
    t.Run("subtest visualization3", func(t *testing.T) {
        assert.True(t, true, "true should be true")
    })
}

go test -v ./...

=== RUN   TestMain
=== RUN   TestMain/subtest_visualization
=== RUN   TestMain/subtest_visualization/sub-subtest_visualization
=== RUN   TestMain/subtest_visualization3
--- PASS: TestMain (0.00s)
    --- PASS: TestMain/subtest_visualization (0.00s)
        --- PASS: TestMain/subtest_visualization/sub-subtest_visualization (0.00s)
    --- PASS: TestMain/subtest_visualization3 (0.00s)
PASS

now say you modified the True assertion method to look something like this.

func True(t TestingT, value bool, msgAndArgs ...interface{}) bool {
    if h, ok := t.(tHelper); ok {
        h.Helper()
    }
    if !value {
        return Fail(t, "Should be true", msgAndArgs...)
    }
    if tt, ok := t.(*testing.T); ok {
        msg := messageFromMsgAndArgs(msgAndArgs...)
        tt.Log(msg)
    }
    return true
}

now your output looks like the following

=== RUN   TestMain
    main_test.go:33: true should be true
=== RUN   TestMain/subtest_visualization
    main_test.go:35: true should be true
=== RUN   TestMain/subtest_visualization/sub-subtest_visualization
    main_test.go:37: true should be true
=== RUN   TestMain/subtest_visualization3
    main_test.go:41: true should be true
--- PASS: TestMain (0.00s)
    --- PASS: TestMain/subtest_visualization (0.00s)
        --- PASS: TestMain/subtest_visualization/sub-subtest_visualization (0.00s)
    --- PASS: TestMain/subtest_visualization3 (0.00s)
PASS

since Go Test doesn't indent the subtest RUN line, i'm not sure if indenting provides much value

=== RUN   TestMain
    main_test.go:33: true should be true
=== RUN   TestMain/subtest_visualization
        main_test.go:35: true should be true
=== RUN   TestMain/subtest_visualization/sub-subtest_visualization
            main_test.go:37: true should be true
=== RUN   TestMain/subtest_visualization3
        main_test.go:41: true should be true
--- PASS: TestMain (0.00s)
    --- PASS: TestMain/subtest_visualization (0.00s)
        --- PASS: TestMain/subtest_visualization/sub-subtest_visualization (0.00s)
    --- PASS: TestMain/subtest_visualization3 (0.00s)
PASS

My current thinking is to let the assertions package do one thing well: check assertions. If passing logs are needed, its easy enough to check the bool returned and use T.Log()