bug: Missing error logs for subtests

hendrikbursian commented 1 month ago

Did you check docs and existing issues?

[X] I have read the documentation.
[X] I have searched the existing issues.

Neovim version (nvim -v)

v0.10.0

Operating system/version

Ubuntu 22.04

Describe the bug

Hi, thanks for the plugin!

I just noticed that the logging of errors for subtests doesn't show up for me as they did a few weeks ago. The commit that is causing the problem for me is e4d8020a9df2883f0cf417d37aaf79a0759a4473. I am using the standard config (without the coverage generation).

Please let me know if you need more information or if its something I overlooked.

Kind regards

Hendrik

    {
        "nvim-neotest/neotest",
        dependencies = {
            "fredrikaverpil/neotest-golang",
            -- commit = "e4d8020a9df2883f0cf417d37aaf79a0759a4473", -- failing commit
            commit = "c9e6fe5de29f28e8198779b24d17e05e69d909dc", -- working commit
            dependencies = {
                {
                    "leoluz/nvim-dap-go",
                    opts = {},
                },
            },
            branch = "main",
        },

        opts = function(_, opts)
            opts.adapters = opts.adapters or {}
            opts.adapters["neotest-golang"] = {
                go_test_args = {
                    "-v",
                    "-race",
                    "-count=1",
                    "-timeout=60s",
                },
                dap_go_enabled = true,
            }

            return opts
        end,
        config = function(_, opts)
            if opts.adapters then
                local adapters = {}
                for name, config in pairs(opts.adapters or {}) do
                    if type(name) == "number" then
                        if type(config) == "string" then
                            config = require(config)
                        end
                        adapters[#adapters + 1] = config
                    elseif config ~= false then
                        local adapter = require(name)
                        if type(config) == "table" and not vim.tbl_isempty(config) then
                            local meta = getmetatable(adapter)
                            if adapter.setup then
                                adapter.setup(config)
                            elseif meta and meta.__call then
                                adapter(config)
                            else
                                error("Adapter " .. name .. " does not support setup")
                            end
                        end
                        adapters[#adapters + 1] = adapter
                    end
                end
                opts.adapters = adapters
            end

            require("neotest").setup(opts)
        end,
    }

Testfile

import (
    "fmt"
    "testing"
)

func TestErrorLogs(t *testing.T) {
    for i := range 3 {
        t.Run(fmt.Sprintf("%d", i), func(t *testing.T) {
            t.Log("log test")
            t.Error("error test")
            t.Fatal("fatal test")
        })
    }
}

Output on commit e4d8020a

=== RUN   TestErrorLogs                                                                                                                                                                                            
  --- FAIL: TestErrorLogs (0.00s)

Output on commit c9e6fe5d (previous commit)

=== RUN   TestErrorLogs
=== RUN   TestErrorLogs/0
    parser_test.go:118: log test
    parser_test.go:119: error test
    parser_test.go:120: fatal test
=== RUN   TestErrorLogs/1
    parser_test.go:118: log test
    parser_test.go:119: error test
    parser_test.go:120: fatal test
=== RUN   TestErrorLogs/2
    parser_test.go:118: log test
    parser_test.go:119: error test
    parser_test.go:120: fatal test
--- FAIL: TestErrorLogs (0.00s)
    --- FAIL: TestErrorLogs/0 (0.00s)
    --- FAIL: TestErrorLogs/1 (0.00s)
    --- FAIL: TestErrorLogs/2 (0.00s)
FAIL
FAIL    github.com/hendrikbursian/w     0.007s

Steps To Reproduce

Use config from my bug report
Use the failing commit hash
Run a failing subtest with t.Run() (see bug report)

See missing error logs

=== RUN   TestErrorLogs                                                                                                                                                                                            
--- FAIL: TestErrorLogs (0.00s)

Expected Behavior

When I run a failing subtest I want to be able to see the log of the failed subtest. For example:

=== RUN   TestErrorLogs
=== RUN   TestErrorLogs/0
    parser_test.go:118: log test
    parser_test.go:119: error test
    parser_test.go:120: fatal test
=== RUN   TestErrorLogs/1
    parser_test.go:118: log test
    parser_test.go:119: error test
    parser_test.go:120: fatal test
=== RUN   TestErrorLogs/2
    parser_test.go:118: log test
    parser_test.go:119: error test
    parser_test.go:120: fatal test
--- FAIL: TestErrorLogs (0.00s)
    --- FAIL: TestErrorLogs/0 (0.00s)
    --- FAIL: TestErrorLogs/1 (0.00s)
    --- FAIL: TestErrorLogs/2 (0.00s)
FAIL
FAIL    github.com/hendrikbursian/w     0.007s

Your Lua setup

{
        "nvim-neotest/neotest",
        dependencies = {
            "fredrikaverpil/neotest-golang",
            -- commit = "e4d8020a9df2883f0cf417d37aaf79a0759a4473", -- failing commit
            commit = "c9e6fe5de29f28e8198779b24d17e05e69d909dc", -- working commit
            dependencies = {
                {
                    "leoluz/nvim-dap-go",
                    opts = {},
                },
            },
            branch = "main",
        },

        opts = function(_, opts)
            opts.adapters = opts.adapters or {}
            opts.adapters["neotest-golang"] = {
                go_test_args = {
                    "-v",
                    "-race",
                    "-count=1",
                    "-timeout=60s",
                },
                dap_go_enabled = true,
            }

            return opts
        end,
        config = function(_, opts)
            if opts.adapters then
                local adapters = {}
                for name, config in pairs(opts.adapters or {}) do
                    if type(name) == "number" then
                        if type(config) == "string" then
                            config = require(config)
                        end
                        adapters[#adapters + 1] = config
                    elseif config ~= false then
                        local adapter = require(name)
                        if type(config) == "table" and not vim.tbl_isempty(config) then
                            local meta = getmetatable(adapter)
                            if adapter.setup then
                                adapter.setup(config)
                            elseif meta and meta.__call then
                                adapter(config)
                            else
                                error("Adapter " .. name .. " does not support setup")
                            end
                        end
                        adapters[#adapters + 1] = adapter
                    end
                end
                opts.adapters = adapters
            end

            require("neotest").setup(opts)
        end,
    }

fredrikaverpil commented 1 month ago

Hey @hendrikbursian and thanks for the super detailed bug report! ⭐

Ah, I think I see the problem here. Right now, each test's output is being filtered so only the exactly one matching test will show its output for each test run. But it does make sense to show all subtest output too, of a given test. Especially when you think how some of sub-tests will be difficult to detect, like this integer iteration one. Just like you experienced, the output of the subtests are not seen anywhere!

I am on vacation currently, and I just whipped up a really quick and dirty fix for this, which I'm not sure is going to be stable enough on its own... but perhaps. I just want to make sure you won't start seeing other unrelated test output for a given test, before merging it in.

Please give it a go if you wish by using neotest-golang with branch = "fix-for-range-test" in your lazy.nvim config.

hendrikbursian commented 1 month ago

Wow, what a turnaround time!

I saw it's already merged into main - its working for me.

Thank you and enjoy your vacation!

fredrikaverpil / neotest-golang