Open qiulaidongfeng opened 8 months ago
cc @golang/tools-team
Fuzz tests are supposed to keep running for some time until a crasher is found. How long do you want to run the tests, when, where, how do you you want results to be present to you?
I hope to run until it fail or vscode is closed.
---Original--- From: "Hyang-Ah Hana @.> Date: Fri, Jan 12, 2024 21:44 PM To: @.>; Cc: @.**@.>; Subject: Re: [golang/go] x/tools/gopls: support run fuzz test (Issue golang/vscode-go#3152)
Fuzz tests are supposed to keep running for some time until a crasher is found. How long do you want to run the tests, when, where, how do you you want results to be present to you?
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Where the result is output. Any possible way is acceptable within vscode.
Background information that may be useful. I created this issue because the first time I clicked "run test" on FuzzTest did go test -run ^FuzzTest$ instead of go test -fuzz ^FuzzTest$ When I need to go test -fuzz ^FuzzTest$ (usually when I finish writing new code to test), I can use the "run package tests" provided by gopls every time go test, and fuzzy test can only start in cmd by itself. This led me to note that it would be easier to use fuzz test if gopls provided a feature to run a specific fuzz test.
the first time I clicked "run test" on FuzzTest did go test -run ^FuzzTest$ instead of go test -fuzz ^FuzzTest$
Oops, that's just a bug! Thanks for reporting. @hyangah this would be a VS Code bug, right? Shall we just roll this into our grand unification of tests?
I personally think it's easier to run fuzz tests from the terminal -- better for browsing output, having terminal history, and cancelling with ctrl-c. But I tend to prefer the terminal more than others. I'm curious to hear if there is support for the idea of running from VS Code.
It's not a bug but intentional behavior for "run test" codelens.
FuzzTest is also a normal test. When it runs with -run
, it runs against the corpus. So the intended workflow is:
I'm curious to hear if there is support for the idea of running from VS Code.
There is a scenario where run fuzz is the last step before git commit, used to check if modifications introduce new bugs, which is more convenient to run in vscode. I encountered this scenario not long ago, and all I need to know is whether run fuzz has not failed. At the same time, I also encountered two fuzz tests sharing the same prefix, and I don't think it's easier to run one fuzz on the terminal than pressing a "run fuzz" in VScode.
At the same time, I also encountered two fuzz tests sharing the same prefix, and I don't think it's easier to run one fuzz on the terminal than pressing a "run fuzz" in VScode.
Just provide the full function name to -fuzz
parameter, not prefix.
The test functionality in vscode go is implemented in the vscode side, not in gopls. LSP doesn't have methods for test execution. Should we move this discussion to vscode?
If it is about vscode, an alternative approach for fuzzing is to utilize the vscode tasks.
At the same time, I also encountered two fuzz tests sharing the same prefix, and I don't think it's easier to run one fuzz on the terminal than pressing a "run fuzz" in VScode.
Just provide the full function name to
-fuzz
parameter, not prefix.
What I'm trying to say is. If the fuzz test involves typing dozens of letters on the command line, it's easier to click "run fuzz".
The test functionality in vscode go is implemented in the vscode side, not in gopls. LSP doesn't have methods for test execution. Should we move this discussion to vscode?
I researched this issue in vscode before opening it "go.useLanguageServer": false The "run test" will be gone "go.useLanguageServer": true "run test" will be available
I researched this issue in vscode before opening it "go.useLanguageServer": false The "run test" will be gone "go.useLanguageServer": true "run test" will be available
Vscode go extension uses gopls to extract test, benchmark, fuzz function names, that are needed to implement the code lens provider. Thus, we disabled the test code lens if the language server is disabled. Other than that the implementation is done by the extension, not gopls.
The code is in https://github.com/golang/vscode-go/blob/c5607ecbf6305f208ae2786834c16240662dc53c/extension/src/goRunTestCodelens.ts, https://github.com/golang/vscode-go/blob/c5607ecbf6305f208ae2786834c16240662dc53c/extension/src/testUtils.ts#L51, etc.
Gopls can help detect fuzz test, but proxying to run fuzz testing seems to be beyond what can be easily offered by LSP spec. We currently have no plan for it.
I think, it's possible that we can consider vscode-go specific UX.
Moving this to the VS Code issue tracker. I don't think this type of workflow is appropriate to run from the language server -- better to invoke the test client side, if desired.
gopls version
Build info ---------- golang.org/x/tools/gopls v0.14.2 golang.org/x/tools/gopls@v0.14.2 h1:sIw6vjZiuQ9S7s0auUUkHlWgsCkKZFWDHmrge8LYsnc= github.com/BurntSushi/toml@v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak= github.com/google/go-cmp@v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= golang.org/x/exp/typeparams@v0.0.0-20221212164502-fae10dda9338 h1:2O2DON6y3XMJiQRAS1UWU+54aec2uopH3x7MAiqGW6Y= golang.org/x/mod@v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0= golang.org/x/sync@v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ= golang.org/x/sys@v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q= golang.org/x/telemetry@v0.0.0-20231114163143-69313e640400 h1:brbkEFfGwNGAEkykUOcryE/JiHUMMJouzE0fWWmz/QU= golang.org/x/text@v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k= golang.org/x/tools@v0.14.1-0.20231114185516-c9d3e7de13fd h1:Oku7E+OCrXHyst1dG1z10etCTxewCHXNFLRlyMPbh3w= golang.org/x/vuln@v1.0.1 h1:KUas02EjQK5LTuIx1OylBQdKKZ9jeugs+HiqO5HormU= honnef.co/go/tools@v0.4.5 h1:YGD4H+SuIOOqsyoLOpZDWcieM28W47/zRO7f+9V3nvo= mvdan.cc/gofumpt@v0.4.0 h1:JVf4NN1mIpHogBj7ABpgOyZc65/UUOkKQFkoURsz4MM= mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc= go: devel go1.22-46410b7 Tue Nov 28 05:13:40 2023 +0000
go env
What did you do?
In vscode , open .go file , it have FuzzTest.
What did you see happen?
I see have "run test" on FuzzTest , it can run go test -run ^FuzzTest$
What did you expect to see?
I see have "run fuzz" on FuzzTest , it can run go test -fuzz ^FuzzTest$ I hope such feature requests will enable gopls to better support fuzz test.
Editor and settings
Logs
No response