x/tools/gopls: file corruption on case-insensitive filesystems

CompeyDev commented 1 year ago

What version of Go, VS Code & VS Code Go extension are you using?

Version Information

* Run `go version` to get version of Go from _the VS Code integrated terminal_. - go version go1.19.3 windows/amd64 * Run `gopls -v version` to get version of Gopls from _the VS Code integrated terminal_. - golang.org/x/tools/gopls v0.10.1 * Run `code -v` or `code-insiders -v` to get version of VS Code or VS Code Insiders. - 1.73.1 * Check your installed extensions to get the version of the VS Code Go extension - 0.36.0 * Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > `Go: Locate Configured Go Tools` command. ``` GOBIN: undefined toolsGopath: gopath: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go GOROOT: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go PATH: C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\dotnet\;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;%SYSTEMROOT%\System32\OpenSSH\;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Python\Python310\Scripts\;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Python\Python310\;%PNPM_HOME%;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Python\Launcher\;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Microsoft\WindowsApps;C:\Users\root.DESKTOP-FTCACMV\.aftman\bin;C:\Users\root.DESKTOP-FTCACMV\.deno\bin;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\JetBrains\Toolbox\scripts;C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Microsoft\WindowsApps;C:\Users\root.DESKTOP-FTCACMV\VSCodium-win32-x64-1.72.2.22289;C:\Users\root.DESKTOP-FTCACMV\VSCodium-win32-x64-1.72.2.22289\bin;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Hyper\resources\bin;C:\Users\root.DESKTOP-FTCACMV\node\node-v18.12.1-win-x64\node_modules\npm\bin;C:\Users\root.DESKTOP-FTCACMV\node\node-v18.12.1-win-x64\node_modules\npm;C:\Users\root.DESKTOP-FTCACMV\node\node-v18.12.1-win-x64;C:\Users\root.DESKTOP-FTCACMV\AppData\Roaming\Spotify;C:\Users\root.DESKTOP-FTCACMV\git;C:\Users\root.DESKTOP-FTCACMV\git\bin; go: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\go.exe: go version go1.19.3 windows/amd64 gotests: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\gotests.exe (version: v1.6.0 built with go: go1.19.3) gomodifytags: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\gomodifytags.exe (version: v1.16.0 built with go: go1.19.3) impl: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\impl.exe (version: v1.1.0 built with go: go1.19.3) goplay: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\goplay.exe (version: v1.0.0 built with go: go1.19.3) dlv: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\dlv.exe (version: v1.9.1 built with go: go1.19.3) golint: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\golint.exe (version: v0.0.0-20210508222113-6edffad5e616 built with go: go1.19.3) gopls: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\gopls.exe (version: v0.10.1 built with go: go1.19.3) go env Workspace Folder (nemo): c:\Users\root.DESKTOP-FTCACMV\dev\nemo set GO111MODULE= set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\root.DESKTOP-FTCACMV\AppData\Local\go-build set GOENV=C:\Users\root.DESKTOP-FTCACMV\AppData\Roaming\go\env set GOEXE=.exe set GOEXPERIMENT= set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GOINSECURE= set GOMODCACHE=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\pkg\mod set GONOPROXY= set GONOSUMDB= set GOOS=windows set GOPATH=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go set GOPRIVATE= set GOPROXY=https://proxy.golang.org,direct set GOROOT=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go set GOSUMDB=sum.golang.org set GOTMPDIR= set GOTOOLDIR=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\pkg\tool\windows_amd64 set GOVCS= set GOVERSION=go1.19.3 set GCCGO=gccgo set GOAMD64=v1 set AR=ar set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD=c:\Users\root.DESKTOP-FTCACMV\dev\nemo\go.mod set GOWORK= set CGO_CFLAGS=-g -O2 set CGO_CPPFLAGS= set CGO_CXXFLAGS=-g -O2 set CGO_FFLAGS=-g -O2 set CGO_LDFLAGS=-g -O2 set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\ROOT~1.DES\AppData\Local\Temp\go-build2785085047=/tmp/go-build -gno-record-gcc-switches ```

Share the Go related settings you have added/edited

None, all default.

Describe the bug

The lint process of the extension is extremely bugged, and ends up deleting random bunches of code. If you save it during this process, the file's contents get discarded entirely.

Steps to reproduce the behavior:

Create a large file, with over 300 lines of code.
Make some changes, and save the file.
If you instantly save the file once you input your code (possibly before the code can be checked by the extension), it discards all the changes made.
If you wait for a minute before saving the file (possibly after the code has been checked by the extension), it removes random characters from your code and saves the file.

Screenshots or recordings

Following are screenshots of before and after saving a file with changes made. Notice how the fmt.Sprint(instanceCheckingErr) I added magically disappears once saved.

Before:

After:

findleyr commented 1 year ago

Hi, I'm not aware of anything in the extension or language server that would cause the behavior you describe. I'm not saying it's impossible, but given the frequency with which you describe encountering this problem, it seems likely that there is something else going on (perhaps a bad interaction with another extension?).

If you are able to reliably repro, we would be very interested to get to the bottom of this. The easiest way for us to see what is going on is to look at LSP logs, which you can collect as described here: https://github.com/golang/vscode-go/blob/master/docs/troubleshooting.md#collect-gopls-information

Could you try to collect those logs for a minimal repro?

CompeyDev commented 1 year ago

Seems like the issue is occurring due to this extension itself, as I disabled all other extensions and this still occurs.

You can view the logs here.

EDIT: This is only occurring with one file, as all other files don't seem to face this. Even tried creating a new VSCode workspace to try and troubleshoot but does not seem to make any difference. This is the exact file.

findleyr commented 1 year ago

Aha, I stared at this for a bit and believe that I have found a likely related issue.

Look at these two logs:

[Trace - 09:54:38.531 AM] Received notification 'textDocument/publishDiagnostics'.
Params: {"uri":"file:///C:/Users/root.DESKTOP-FTCACMV/dev/nemo/src/server/networking.go","diagnostics":[{"range":{"start":{"line":85,"character":15},"end":{"line":85,"character":33}},"severity":1,"code":"UndeclaredName","codeDescription":{"href":"https://pkg.go.dev/golang.org/x/tools/internal/typesinternal#UndeclaredName"},"source":"compiler","message":"undeclared name: jsonErrorResponse2"}]}

[Trace - 09:54:38.531 AM] Received notification 'textDocument/publishDiagnostics'.
Params: {"uri":"file:///C:/Users/root.DESKTOP-FTCACMV/dev/nemo/src/Server/networking.go","diagnostics":[]}

I noticed that we apparently have two different representations for this file: one with a capital 'S' for 'Server', and another with a lower-case 's' for 'server'. It looks like the 'didChangeWatchedFile' notification (arriving from the file watcher) has a lower-case 's', but notifications related to the open file have upper-case 'S'. Clearly gopls is confused about the identity of this file, because it sent different diagnostics for the two files.

I'm not yet sure how this causes the corruption you observe, but it seems likely that either the editor is confused about file identity, gopls is confused, or both.

Can you tell me more about your environment? What does the filesystem say about this path? Do you know where the capital 'S' originated?

I'll do more digging to try to understand how this could cause problems in gopls.

findleyr commented 1 year ago

It would also be incredibly helpful if you could include a full set of logs for a session that repros the problem, starting from the beginning. That would let us see when and how the two different casings were communicated.

findleyr commented 1 year ago

Hi, I believe I understand this now. The key is in the following logs:

[Trace - 09:54:37.094 AM] Sending request 'textDocument/codeAction - (115)'.
Params: {"textDocument":{"uri":"file:///c%3A/Users/root.DESKTOP-FTCACMV/dev/nemo/src/Server/networking.go"},"range":{"start":{"line":85,"character":33},"end":{"line":85,"character":33}},"context":{"diagnostics":[],"triggerKind":2}}

[Trace - 09:54:37.096 AM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2022/12/02 09:54:37 fixImports(filename=\"C:\\\\Users\\\\root.DESKTOP-FTCACMV\\\\dev\\\\nemo\\\\src\\\\server\\\\networking.go\"), abs=\"C:\\\\Users\\\\root.DESKTOP-FTCACMV\\\\dev\\\\nemo\\\\src\\\\server\\\\networking.go\", srcDir=\"C:\\\\Users\\\\root.DESKTOP-FTCACMV\\\\dev\\\\nemo\\\\src\\\\server\" ...\n"}

[Trace - 09:54:37.097 AM] Received response 'textDocument/codeAction - (115)' in 2ms.
Result: null

These show that gopls got a request for Server, but is actually looking for server. That means that gopls has canonicalized the URL internally to the version on disk (it uses the first version it sees as canonical).

But it looks like content invalidation does not canonicalize URIs. So a didOpen or didChange of the same URI will NOT invalidate on-disk contents inside of gopls. This is simply a bug.

In other words, I think the bug goes as follows:

Load a workspace with server on a case insensitive file-system.
Issue a didOpen and didChange for Server. This will fail to invalidate the content already loaded for server.
Issue a formatting request for Server. Gopls will canonicalize this request to server, and find the old contents (which have not been invalidated). If the resulting edits are non-empty, they will be incorrect and leave a corrupted state in the editor.

Now to find a case-insensitive filesystem to reproduce...

CompeyDev commented 1 year ago

This seems correct. I did a bit of digging too, and attempted to copy and paste the contents of the file into another. This does not repro the issue, but if I directly copy the file itself (and not just its contents), the issue does reoccur, no matter where I store it.

CompeyDev commented 1 year ago

It would also be incredibly helpful if you could include a full set of logs for a session that repros the problem, starting from the beginning. That would let us see when and how the two different casings were communicated.

I created a new workspace in the same folder and opened the db.go file, followed by the buggy networking.go file, and these are the logs it produced.

findleyr commented 1 year ago

Thanks, while we haven't yet reproduced locally, I think we have enough information about the nature of the bug. I have transferred this to the Go issue tracker.

CompeyDev commented 1 year ago

@findleyr Any updates?

findleyr commented 1 year ago

@CompeyDev unfortunately this is going to take a little while to fix. We have ideas, but it is a rather low level bug. We will try to fix this for the next minor release, which is slated for ~February, but there is no guarantee.

CompeyDev commented 1 year ago

@CompeyDev unfortunately this is going to take a little while to fix. We have ideas, but it is a rather low level bug. We will try to fix this for the next minor release, which is slated for ~February, but there is no guarantee.

Understood, is there at least a way I can mitigate this? It is quite disruptive at the moment.

findleyr commented 1 year ago

@CompeyDev sorry for missing your last comment. If you can get your editor to agree about file casing that would avoid the problem.

I'm working on a fix now, but as I said it will not arrive this month. I will ping here when there is something ready for testing.

CompeyDev commented 1 year ago

If you can get your editor to agree about file casing that would avoid the problem.

How exactly can I achieve this in VSCode? I'm not really sure what kind of casing VSCode on Windows follows.

gopherbot commented 1 year ago

Change https://go.dev/cl/462819 mentions this issue: gopls: fix windows file corruption

findleyr commented 1 year ago

How exactly can I achieve this in VSCode? I'm not really sure what kind of casing VSCode on Windows follows.

Sorry, I'm not sure why VS Code is choosing a different spelling of this file.

The CL above (not yet submitted) fixes the file corruption, but there is more work yet as other features won't work in this file: we need to identify that this is the same file as returned to us from the go command.

CompeyDev commented 1 year ago

Heya, it's been a while now - any new updates, @findleyr?

findleyr commented 1 year ago

@CompeyDev sorry, missed your ping.

We have been busy with the v0.12.0 release, but this is one of the largest issues we hope to fix for v0.13.0, as we believe it is the largest outstanding gopls bug.

CompeyDev commented 1 year ago

Alright, as for now I've switched my environment to Linux & everything seems to work as expected.

Thanks for the update!

alesji commented 10 months ago

I am facing the same issue right now... can i at least somehow restart/clear gopls completely so it stops doing it? it happened when i renamed User.go to user.go on windows... I guess it has User.go stored somewhere, so some sort of clearing would help, how do i do that?

edit clearing c:/Users/%USER%/AppData/Local/gopls/ helped

findleyr commented 10 months ago

@alesji are you experiencing corruption, or just a broken LSP server?

Your files should no longer be corrupted.

golang / go