Failure to debug a suite test that is in a different file than the caller test

[nirhaas]

What version of Go, VS Code & VS Code Go extension are you using?

Version Information

* Run `go version` to get version of Go from _the VS Code integrated terminal_. ``` go version go1.18.1 darwin/arm64 ``` * Run `gopls -v version` to get version of Gopls from _the VS Code integrated terminal_. ``` Build info ---------- golang.org/x/tools/gopls v0.9.3 golang.org/x/tools/gopls@v0.9.3 h1:wfh4cfJAKwOG49sCE4ldafYeD5rlejE/gJQ7JAR5yX0= github.com/BurntSushi/toml@v1.2.0 h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0= github.com/google/go-cmp@v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg= github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= golang.org/x/exp/typeparams@v0.0.0-20220722155223-a9213eeb770e h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE= golang.org/x/mod@v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s= golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw= golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s= golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk= golang.org/x/tools@v0.1.13-0.20220811140653-b901dff69f70 h1:yg8cGxL0g/WIlkF25aKuIAZacqXROXw4Dt57iKLWg8w= golang.org/x/vuln@v0.0.0-20220725105440-4151a5aca1df h1:BkeW9/QJhcigekDUPS9N9bIb0v7gPKKmLYeczVAqr2s= honnef.co/go/tools@v0.3.2 h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34= mvdan.cc/gofumpt@v0.3.1 h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8= mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc= go: go1.18.1 ``` * Run `code -v` or `code-insiders -v` to get version of VS Code or VS Code Insiders. ``` 1.70.1 6d9b74a70ca9c7733b29f0456fd8195364076dda arm64 ``` * Check your installed extensions to get the version of the VS Code Go extension ``` v0.35.1 ``` * Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > `Go: Locate Configured Go Tools` command. ``` Checking configured tools.... GOBIN: undefined toolsGopath: gopath: /Users/nhaas/go GOROOT: /opt/homebrew/Cellar/go/1.18.1/libexec PATH: /Users/nhaas/go/bin:/Users/nhaas/Library/Python/3.8/bin:/opt/homebrew/opt/libpq/bin:/Users/nhaas/.tgenv/bin:/Users/nhaas/.tfenv/bin:/opt/homebrew/opt/make/libexec/gnubin:/opt/homebrew/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Users/nhaas/.cargo/bin:/Users/nhaas/.fzf/bin go: /opt/homebrew/bin/go: go version go1.18.1 darwin/arm64 gotests: /Users/nhaas/go/bin/gotests (version: v1.6.0 built with go: go1.18.1) gomodifytags: /Users/nhaas/go/bin/gomodifytags (version: v1.16.0 built with go: go1.18.1) impl: /Users/nhaas/go/bin/impl (version: v1.1.0 built with go: go1.18.1) goplay: /Users/nhaas/go/bin/goplay (version: v1.0.0 built with go: go1.18.1) dlv: /Users/nhaas/go/bin/dlv (version: v1.8.3 built with go: go1.18.1) golangci-lint: /Users/nhaas/go/bin/golangci-lint (version: v1.48.0 built with go: go1.18.1) gopls: /Users/nhaas/go/bin/gopls (version: v0.9.3 built with go: go1.18.1) go env Workspace Folder (vscode-go): /Users/nhaas/Temp/vscode-go GO111MODULE="" GOARCH="arm64" GOBIN="" GOCACHE="/Users/nhaas/Library/Caches/go-build" GOENV="/Users/nhaas/Library/Application Support/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="arm64" GOHOSTOS="darwin" GOINSECURE="" GOMODCACHE="/Users/nhaas/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/nhaas/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/opt/homebrew/Cellar/go/1.18.1/libexec" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/opt/homebrew/Cellar/go/1.18.1/libexec/pkg/tool/darwin_arm64" GOVCS="" GOVERSION="go1.18.1" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/nhaas/Temp/vscode-go/go.mod" GOWORK="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/t8/83vffqv11_bbfxbbdc3fsfx40000gn/T/go-build4218516277=/tmp/go-build -gno-record-gcc-switches -fno-common" ```

Share the Go related settings you have added/edited

  "go.lintTool": "golangci-lint",
  "go.formatTool": "gosimports",
  "go.formatFlags": ["-local", "github.com/company"],
  "gopls": {
    "ui.semanticTokens": true
  },

Describe the bug

Terminology: When using testify suite, there is

1. The suite itself (struct)
2. The suite tests (test functions)
3. A normal test that runs suite.Run with the suite struct.

Following #1130, when clicking the button to "Debug test" for a suite test that is in a different file than the "normal test", we never stop at the breakpoint, and get testing: warning: no tests to run printed on the debug console.

I suspect that the reason for this behavior is that the function findAllTestSuiteRuns (https://github.com/golang/vscode-go/blob/d67af7f9d5eb5f189d4d20a9c0df580742be73c6/src/testUtils.ts#L218) only has access to the document symbols when trying to figure out whether there is a suite.Run. If the "normal test" is in a different file (which is different document), it returns an empty array, which would later be parsed to a wrong -test.run flag to dlv.

From dlv log output (note that ^$):

2022-08-15T21:51:57+03:00 debug layer=dap parsed launch config: {
    "mode": "test",
    "program": ".",
    "args": [
        "-test.run",
        "^$",
        "-testify.m",
        "^TestSomething$"
    ],
    "env": {
        "GOPATH": "/Users/nhaas/go"
    },
    "backend": "default",
    "stackTraceDepth": 50
}

Steps to reproduce the behavior:

Follow the steps of #1130 and try to click "Debug test" for a suite test in the other file.

[gopherbot]

Change https://go.dev/cl/424094 mentions this issue: fix multifile suite test fails to debug

[alankritjoshi]

I’ve been facing this issue for the last two weeks.

A workaround for me for now was to begin debug test from the file where the test suite is declared or use the Testing side panel.

Thanks for identifying it and the fix!

[divmgl]

A workaround for me for now was to begin debug test from the file

This actually doesn't work for me at all. Running debug test on individual Golang tests for files that are outside of where the test suite is declared simply doesn't work.

The only fix I've found is to create a test suite per file using the main test suite embedded.

For instance, say I have a test suite in main_test.go named APISuite, in tenant_test.go I have to use:

type TenantSuite struct {
    APISuite
}

func TestTenantSuite(t *testing.T) {
    suite.Run(t, new(TenantSuite))
}

func (s *TenantSuite) TestUserHasTenant() {
    accessToken := s.registerUser()

Unfortunate that this issue is still happening. It's been several years.

[Altiano]

I just hope they prioritize this one..

[shimonacarvalho]

Also having this issue in my tests.

[divmgl]

It does look like there's some movement on a fix for this: https://github.com/golang/vscode-go/pull/2415#issuecomment-1265184507

[esdrasbeleza]

Also having this issue :(

[denis-sazonov-gaembla]

doesn't work for me as well :(

[jdahm]

I want to use vscode-go for developing where possible, but I use this pattern too often to make this a smooth experience. Really looking forward to this getting resolved soon.

[hendrics]

is there a work around for this? Or there's no way to debug a test function from viscose? What folks do instead?

[piavgh]

is there a work around for this? Or there's no way to debug a test function from viscose? What folks do instead?

Same, I really want to remove Goland from my development environment (it eats up too much system resource), but the developer experience in Goland is so great (especially in this individual test case).

[MortenGerdes]

Would love to see this fixed. I've also been using this testing suite quite a lot and have tilted towards using Goland (or IntelliJ with Go plugin) because it just allowed for debugging this so seemless.

Is there anything we can do to see the fix for this merged?

[makeitraina]

Would love to see this fixed! Any update?

[gopherbot]

Change https://go.dev/cl/555676 mentions this issue: src/goTest: fix multifile suite test fails to debug

[morremeyer]

Just verified that this is fixed in the nightly¹ version. Thanks @Cr4zySheep for taking this on!

This makes my (and I assume many others) dev experiences much more smooth.

¹If you want to get this asap, disable the Go extension and install Go Nightly instead

[matsaleh13]

I'm running v0.41.4 of the extension and this problem is happening for me now. I saw from the changelog that the fix was applied to v0.41.2, however. Any chance there's been a regression? AFAIK, I shouldn't have to do anything special to make use of this fix, right? Thanks.

[marcelo-rocha]

I'm running v0.41.4 of the extension and this problem is happening for me now. I saw from the changelog that the fix was applied to v0.41.2, however. Any chance there's been a regression? AFAIK, I shouldn't have to do anything special to make use of this fix, right? Thanks.

I've noticed the test function that creates the test suite cannot have anything else than the test suite creation itself, so I suggest checking if your function is like this:

func TestMySuite(t *testing.T) {
    suite.Run(t, new(MySuite))
}

[matsaleh13]

I've noticed the test function that creates the test suite cannot have anything else than the test suite creation itself, so I suggest checking if your function is like this:

Oh wow... that's very specific, isn't it?

My code (not working):

func TestRunSuite(t *testing.T) {
    batchSuite := new(KVDatastoreSuite)
    suite.Run(t, batchSuite)
}

... and now, fixed:

func TestRunSuite(t *testing.T) {
    suite.Run(t, new(KVDatastoreSuite))
}

Thanks very much for that suggestion! I would never have thought of that.

I guess it makes sense that the extension would have to do some kind of code parsing to be able to figure out what's going on, right?

Thanks again!

[adityachowdhry-probo]

Having the same issue on version v0.41.4

My test suite structure looks like this

func TestMySuite(t *testing.T){
  t.Run("TestA", func(t *testing.T){
    t.Run("TestASubTest", func(t *testing.T){
    })
  })
  t.Run("TestB", func(t *testing.T){
  })
}

Am I doing something wrong here?

[firelizzard18]

@adityachowdhry-probo This issue is specifically about test suites built with stretchr. Generic subtest support is tracked in #2445.