proposal: go/scanner: stop adding source file directory to relative file name in //line comment

podtserkovskiy commented 2 weeks ago

Proposal Details

The current behaviour

The go/scanner.Scanner.updateLineInfo prepends name of *.cgo1.go file directory to the relative path of the original CGo file taken from //line comment.

For example:

$ go tool cgo -objdir cgo_objdir -trimpath $(pwd) src/test1.go 
# Generates file `cgo_objdir/test1.cgo1.go`
# The file have line: `//line src/test1.go:1:1`

Let's parse it then with parser.ParseFile(fset, "cgo_objdir/test1.cgo1.go", nil, parser.ParseComments)and check fset value.

The issue is that fset will contain go/token.File with infos[0].lineInfo.Filename = "cgo_objdir/src/test1.go".

The path "cgo_objdir/src/test1.go" in lineInfo.Filename is invalid.

There was an attempt to fix this behaviour, but it was rollbacked in CL 127658 due to Issue 26671

When this becomes a problem?

In build systems like Buck2 or Bazel the build actions may be executed on different hosts, so we can't bake absolute paths inside build artifacts.

We have a way to prevent this by adding -trimpath $(pwd) to build commands. However the go/scanner will compute invalid filepath in this case as shown in the above.

A specific example

The cgocall analyser in golang.org/x/tools is trying to access the original source file by fset.Position(raw.Pos()).Filename, but fails when relative filepaths used because the filename is invalid.

This doesn't allow seamless integration of Buck2/Bazel with code analysis tools.

Possible solutions

Option 1: Bold

Rollback CL 127658, but this probably can break some existing linters or other programs like it was in Issue 26671.

Option 2: Safe

Rollback CL 127658 and add a GODEBUG constant to minimise potential damage to existing programs.

golineprependdir = "1" - old behaviour
golineprependdir != "1" - new behaviour

I'm not sure what the default behaviour of should be for go1.24. I'm happy with any option as we can set GODEBUG env-var before running code analysers.

gabyhelp commented 2 weeks ago

Related Issues and Documentation

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

ianlancetaylor commented 2 weeks ago

CC @griesemer @findleyr

podtserkovskiy commented 1 week ago

I've created a test project podtserkovskiy/issue-69689 with go and cgo testcases for popular linters to make sure linters work for the same before and after my change. I've reverted CL 127658 on Go release-branch.go1.23 and ran popular linters golangci_lint (with all lints enabled including unparam, unconvert and unused) and staticcheck on podtserkovskiy/issue-69689.

I compared results against the linters compiled with non-modified Go. I downloaded the linters from Homebrew repository.

`golangci_lint` before my changes

issue-69689 git:(main) golangci-lint run --enable-all
WARN The linter 'execinquery' is deprecated (since v1.58.0) due to: The repository of the linter has been archived by the owner.  
WARN The linter 'exportloopref' is deprecated (since v1.60.2) due to: Since Go1.22 (loopvar) this linter is no longer relevant. Replaced by copyloopvar. 
WARN The linter 'gomnd' is deprecated (since v1.58.0) due to: The linter has been renamed. Replaced by mnd. 
rand/rand.go:11:18: unused-parameter: parameter 'j' seems to be unused, consider removing or renaming it as _ (revive)
func Seed(i int, j int) { // unparam
                 ^
rand/rand.go:12:9: unnecessary conversion (unconvert)
        i = int(i) // unconvert
               ^
rand/rand.go:9:6: func `unusedFunc` is unused (unused)
func unusedFunc() {} // unused func
     ^
rand/rand.go:14:13: SA1004: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (staticcheck)
        time.Sleep(1)
                   ^
rand_cgo/rand_cgo.go:15:18: unused-parameter: parameter 'j' seems to be unused, consider removing or renaming it as _ (revive)
func Seed(i int, j int) { // unparam
                 ^
rand_cgo/rand_cgo.go:16:9: unnecessary conversion (unconvert)
        i = int(i) // unconvert
               ^
rand_cgo/rand_cgo.go:18:13: SA1004: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (staticcheck)
        time.Sleep(1) // SA1004
                   ^

`golangci_lint` after my changes

➜  issue-69689 git:(main) GOROOT=$PWD/../go PATH=$PWD/../go/bin:$PATH ./../golangci-lint/golangci-lint run --enable-all
WARN The linter 'exportloopref' is deprecated (since v1.60.2) due to: Since Go1.22 (loopvar) this linter is no longer relevant. Replaced by copyloopvar. 
WARN The linter 'gomnd' is deprecated (since v1.58.0) due to: The linter has been renamed. Replaced by mnd. 
WARN The linter 'execinquery' is deprecated (since v1.58.0) due to: The repository of the linter has been archived by the owner.  
rand/rand.go:11:18: unused-parameter: parameter 'j' seems to be unused, consider removing or renaming it as _ (revive)
func Seed(i int, j int) { // unparam
                 ^
rand/rand.go:12:9: unnecessary conversion (unconvert)
        i = int(i) // unconvert
               ^
rand/rand.go:9:6: func `unusedFunc` is unused (unused)
func unusedFunc() {} // unused func
     ^
rand/rand.go:14:13: SA1004: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (staticcheck)
        time.Sleep(1)
                   ^
rand_cgo/rand_cgo.go:15:18: unused-parameter: parameter 'j' seems to be unused, consider removing or renaming it as _ (revive)
func Seed(i int, j int) { // unparam
                 ^
rand_cgo/rand_cgo.go:16:9: unnecessary conversion (unconvert)
        i = int(i) // unconvert
               ^
rand_cgo/rand_cgo.go:18:13: SA1004: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (staticcheck)
        time.Sleep(1) // SA1004
                   ^

`staticcheck` before my changes

➜  issue-69689 git:(main) staticcheck ./...
rand/rand.go:9:6: func unusedFunc is unused (U1000)
rand/rand.go:14:13: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (SA1004)
rand_cgo/rand_cgo.go:18:13: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (SA1004)

`staticcheck` after my changes

➜  issue-69689 git:(main) GOROOT=$PWD/../go PATH=$PWD/../go/bin:$PATH /tmp/staticcheck ./...
rand/rand.go:9:6: func unusedFunc is unused (U1000)
rand/rand.go:14:13: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (SA1004)
rand_cgo/rand_cgo.go:18:13: sleeping for 1 nanoseconds is probably a bug; be explicit if it isn't (SA1004)

Conclusion

As we can see from these examples #26671 doesn't reproduce on the linters anymore if we rollback CL 127658. Therefore, it should be reasonable to change go/scanner.Scanner.updateLineInfo behaviour by removing CL 127658

However, there's a chance we may break some unknown to me existing programmes. I think it's reasonable to add a GODEBUG option to enable existing programs temporary opt-out from the go/scanner behaviour change.

Since the blast-radius shouldn't be large I think we can remove this GODEBUG option in go1.26 as likely nobody would need it by this time.

I'll make and publish a CL shortly.

gopherbot commented 1 week ago

Change https://go.dev/cl/618276 mentions this issue: go/scanner: Preserve relative filenames from //line comments.

golang / go