gopls docs need updates for bzlmod

[jmhodges]

What version of rules_go are you using?

0.50.1

What version of gazelle are you using?

0.39.0

What version of Bazel are you using?

7.3.1

Does this issue reproduce with the latest releases of all the above?

yes

What operating system and processor architecture are you using?

macOS 15.0

Any other potentially useful information about your toolchain?

gopls v0.16.2 (and go 1.23.1 on the host)

What did you do?

Try to follow the documentation on https://github.com/bazelbuild/rules_go/wiki/Editor-setup and specifically for vscode

What did you expect to see?

When using bzlmod, the Editor setup documentation should get you to a working state for vscode.

What did you see instead?

First, the example gopackagesdriver.sh references a @io_bazel_rules_go that doesn't exist. (I believe the command should be @rules_go//go/tools/gopackagesdriver)

Second, the vscode specific section says to set:

  "go.goroot": "${workspaceFolder}/bazel-${workspaceFolderBasename}/external/go_sdk",

However, this directory doesn't exist. I'm not sure what to set it to.

I tried setting to:

  "go.goroot": "${workspaceFolder}/bazel-${workspaceFolderBasename}/external/rules_go~~go_sdk~mtganalysis__download_0/",

but that got me this error:

the packages.Load error

``` initialization failed: packages.Load error: /Users/jmhodges/src/github.com/jmhodges/mtganalysis/tools/gopackagesdriver.sh: exit status 1: Computing main repo mapping: Loading: Loading: 0 packages loaded Analyzing: target @@rules_go~//go/tools/gopackagesdriver:gopackagesdriver (0 packages loaded, 0 targets configured) Analyzing: target @@rules_go~//go/tools/gopackagesdriver:gopackagesdriver (0 packages loaded, 0 targets configured) [0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt INFO: Repository rules_go~~go_sdk~mtganalysis__host_0 instantiated at: : in Repository rule go_host_sdk_rule defined at: /private/var/tmp/_bazel_jmhodges/e8429118da23e313ae2b83f19bf3f5f4/external/rules_go~/go/private/sdk.bzl:30:35: in ERROR: An error occurred during the fetch of repository 'rules_go~~go_sdk~mtganalysis__host_0': Traceback (most recent call last): File "/private/var/tmp/_bazel_jmhodges/e8429118da23e313ae2b83f19bf3f5f4/external/rules_go~/go/private/sdk.bzl", line 25, column 36, in _go_host_sdk_impl platform = _detect_sdk_platform(ctx, goroot) File "/private/var/tmp/_bazel_jmhodges/e8429118da23e313ae2b83f19bf3f5f4/external/rules_go~/go/private/sdk.bzl", line 562, column 13, in _detect_sdk_platform fail("Could not detect SDK platform: failed to find " + str(path)) Error in fail: Could not detect SDK platform: failed to find /Users/jmhodges/src/github.com/jmhodges/mtganalysis/bazel-mtganalysis/external/rules_go~~go_sdk~mtganalysis__download_0/pkg/tool ERROR: /private/var/tmp/_bazel_jmhodges/e8429118da23e313ae2b83f19bf3f5f4/external/rules_go~/go/tools/gopackagesdriver/BUILD.bazel:24:10: While resolving toolchains for target @@rules_go~//go/tools/gopackagesdriver:gopackagesdriver (096dcc8): invalid registered toolchain '@go_toolchains//:all': while parsing '@go_toolchains//:all': no such package '@@rules_go~~go_sdk~mtganalysis__host_0//': Could not detect SDK platform: failed to find /Users/jmhodges/src/github.com/jmhodges/mtganalysis/bazel-mtganalysis/external/rules_go~~go_sdk~mtganalysis__download_0/pkg/tool ERROR: Analysis of target '@@rules_go~//go/tools/gopackagesdriver:gopackagesdriver' failed; build aborted INFO: Elapsed time: 0.333s, Critical Path: 0.01s INFO: 1 process: 1 internal. ERROR: Build did NOT complete successfully FAILED: ERROR: Build failed. Not running target ```

Bafflingly, when I set that directory in place, the entire bazel-mtganalysis/external directory seems to be removed? I'm not sure what's happening there.

[fmeum]

Those fixes look right. Could you perhaps send a PR to update the docs?

The better alternative to hardcoding the Bzlmod repo path is https://github.com/buildbuddy-io/bazel_env.bzl, which also gives you a hermetic go binary in your PATH that uses @rules_go//go.

If the symlink ends up being deleted, the most likely cause is that something in your .bazelrc isn't picked up by the bazel query commands issued by the package driver.

[jmhodges]

Hm, I have no .bazelrc in this repo.

[jmhodges]

I'll post a repro, one sec

[jmhodges]

Okay, I have a repro in this branch of my bazel_bugs repo with a README describing how to use it: https://github.com/jmhodges/bazel_bugs/tree/vscode-gopls-4124

For clarity: this happens even with a vscode installation that has only the Go extension installed

[jmhodges]

Also, I don't understand how to use bazel_env to get me a goroot setting I can use in vscode. vscode doesn't know about direnv out of the box.

[jmhodges]

Yep, the external directory absolutely is removed when you hit "Reload Window" in vscode. I had run bazel build //foo/..., and then ran the "Reload Window" on the empty, Go-extension-only profile in vscode and the directory is deleted.

Edited to add more details: If I "Reload Window" in a non-Go file, and then switch to the foo/foo.go file, the deletion of the external directory happens on that file opening up. So, it seems to be happening on load of either gopackagesdriver, the Go vscode extension, or gopls. I'm not sure which.

[fmeum]

Also, I don't understand how to use bazel_env to get me a goroot setting I can use in vscode. vscode doesn't know about direnv out of the box.

direnv is only relevant for adding binaries to PATH, bazel_env will give you stable directories with friendly names for your toolchains even without direnv.

I will take a look at the repo.

[fmeum]

I can't repro this with the instructions you provided. Is it possible that home or system rc files get in the way? Could you add --announce_rc to the project's .bazelrc file and check the output?

[jmhodges]

I don't see any differences in output in the gopls logs in vscode nor in local runs of the commands.

Here's a full gist of the gopls server logs.

[jmhodges]

Oh, I typoed. Here's the logs with the announce_rc in place: https://gist.github.com/jmhodges/33ae0bb4f631b3250ac103defc95592e

(I've also pushed the .bazelrc change up so you can see I did it correctly)

[jmhodges]

I made the repro instructions a bit more clear. I understand if you can't repro after them that I'll need to debug this myself. There's a bit of waiting involved. Maybe this is additionally a zsh or macOS or homebrew problem? I'm not sure what platform you're on.

Could def be a me problem, but it would be very strange.

[jmhodges]

I made the external directory immutable just to see what would happen (with chflags uchg /private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/execroot/_main/external), and now I see the execroot/_main directory having far fewer files in it and gopls logs erroring out saying:

ERROR: Failed to prepare the symlink forest: com.google.devtools.build.lib.unix.FilePermissionException: fchmod (/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/execroot/_main/external) (Operation not permitted)

The file listing before loading up vscode (and after running bazel build //foo/...):

$  ls -l /private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/execroot/_main/
total 0
lrwxr-xr-x  1 jmhodges  wheel   62 Sep 30 01:34 BUILD.bazel -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/BUILD.bazel
lrwxr-xr-x  1 jmhodges  wheel   63 Sep 30 01:34 MODULE.bazel -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/MODULE.bazel
lrwxr-xr-x  1 jmhodges  wheel   68 Sep 30 01:34 MODULE.bazel.lock -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/MODULE.bazel.lock
lrwxr-xr-x  1 jmhodges  wheel   60 Sep 30 01:34 README.md -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/README.md
drwxr-xr-x  9 jmhodges  wheel  288 Sep 30 01:34 bazel-out
drwxr-xr-x  9 jmhodges  wheel  288 Sep 30 01:34 external
lrwxr-xr-x  1 jmhodges  wheel   54 Sep 30 01:34 foo -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/foo
lrwxr-xr-x  1 jmhodges  wheel   57 Sep 30 01:34 go.mod -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/go.mod
lrwxr-xr-x  1 jmhodges  wheel   56 Sep 30 01:34 tools -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/tools
lrwxr-xr-x  1 jmhodges  wheel   71 Sep 30 01:34 vscode-settings.json -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/vscode-settings.json

The file listing after loading up foo/foo.go in vscode:

$  ls -l /private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/execroot/_main/
total 0
lrwxr-xr-x  1 jmhodges  wheel   63 Sep 30 01:34 MODULE.bazel -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/MODULE.bazel
lrwxr-xr-x  1 jmhodges  wheel   68 Sep 30 01:34 MODULE.bazel.lock -> /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/MODULE.bazel.lock
drwxr-xr-x  9 jmhodges  wheel  288 Sep 30 01:34 bazel-out
drwxr-xr-x  9 jmhodges  wheel  288 Sep 30 01:34 external

[jmhodges]

I think the problem is setting GOROOT.

If I run the command to set GOROOT:

$ unset GOROOT # Just in case
$ bazel build //foo/...
$ export GOROOT="$(PWD)/bazel-bazel_bugs/external/rules_go~~go_sdk~bazel_bugs__download_0"
$ bazel build //foo/...

I get errors in my terminal on the second run of bazel build //foo/.. that are the same as I did in vscode.

Specifically:

INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=151
INFO: Reading rc options for 'build' from /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/.bazelrc:
  Inherited 'common' options: --announce_rc
INFO: Repository rules_go~~go_sdk~bazel_bugs__host_0 instantiated at:
  <builtin>: in <toplevel>
Repository rule go_host_sdk_rule defined at:
  /private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl:30:35: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_go~~go_sdk~bazel_bugs__host_0':
   Traceback (most recent call last):
    File "/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl", line 25, column 36, in _go_host_sdk_impl
        platform = _detect_sdk_platform(ctx, goroot)
    File "/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl", line 562, column 13, in _detect_sdk_platform
        fail("Could not detect SDK platform: failed to find " + str(path))
Error in fail: Could not detect SDK platform: failed to find /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/bazel-bazel_bugs/external/rules_go~~go_sdk~bazel_bugs__download_0/pkg/tool
ERROR: <builtin>: fetching go_host_sdk_rule rule //:rules_go~~go_sdk~bazel_bugs__host_0: Traceback (most recent call last):
    File "/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl", line 25, column 36, in _go_host_sdk_impl
        platform = _detect_sdk_platform(ctx, goroot)
    File "/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl", line 562, column 13, in _detect_sdk_platform
        fail("Could not detect SDK platform: failed to find " + str(path))
Error in fail: Could not detect SDK platform: failed to find /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/bazel-bazel_bugs/external/rules_go~~go_sdk~bazel_bugs__download_0/pkg/tool
ERROR: /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/foo/BUILD.bazel:3:11: While resolving toolchains for target //foo:lib (7170974): invalid registered toolchain '@go_toolchains//:all': while parsing '@go_toolchains//:all': no such package '@@rules_go~~go_sdk~bazel_bugs__host_0//': Could not detect SDK platform: failed to find /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/bazel-bazel_bugs/external/rules_go~~go_sdk~bazel_bugs__download_0/pkg/tool
ERROR: Analysis of target '//foo:lib' failed; build aborted
INFO: Elapsed time: 0.176s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

[jmhodges]

Like, what if setting GOROOT now causes bazel or rules_go to say "Oh, dang, we have to rebuild our whole environment to accommodate this new env var", but doing so still depends on something Go-y that didn't realize the whole bazel setup was blown away, and when it's asked to do work, sees that it doesn't have what it needs and crashes?

[jmhodges]

Ah ha, and if I uninstall my host machine's go binary, and then try to run the bazel build //foo/... with no GOROOT set, it fails because _detect_host_sdk fails to find it. I'm not sure why this happens. It looks to be when we're fetching the jdk for bazel?

It really does seem like there's some very important dep on the host machine's Go that setting a GOROOT over could cause problems in interesting ways.

Here's the error I saw:

The _detec_host_sdk error

``` $ which go go not found $ unset GOROOT $ bazel build //foo/... INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=151 INFO: Reading rc options for 'build' from /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/.bazelrc: Inherited 'common' options: --announce_rc INFO: Repository rules_go~~go_sdk~bazel_bugs__host_0 instantiated at: : in Repository rule go_host_sdk_rule defined at: /private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl:30:35: in ERROR: An error occurred during the fetch of repository 'rules_go~~go_sdk~bazel_bugs__host_0': Traceback (most recent call last): File "/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl", line 24, column 30, in _go_host_sdk_impl goroot = _detect_host_sdk(ctx) File "/private/var/tmp/_bazel_jmhodges/eb16c8c3f4af4addfaf08c38f6204969/external/rules_go~/go/private/sdk.bzl", line 553, column 13, in _detect_host_sdk fail("Could not detect host go version") Error in fail: Could not detect host go version ERROR: /Users/jmhodges/src/github.com/jmhodges/bazel_bugs/foo/BUILD.bazel:3:11: While resolving toolchains for target //foo:lib (096dcc8): invalid registered toolchain '@go_toolchains//:all': while parsing '@go_toolchains//:all': no such package '@@rules_go~~go_sdk~bazel_bugs__host_0//': Could not detect host go version ERROR: Analysis of target '//foo:lib' failed; build aborted INFO: Elapsed time: 0.115s, Critical Path: 0.00s INFO: 1 process: 1 internal. ERROR: Build did NOT complete successfully FAILED: Fetching repository @@rules_java~~toolchains~local_jdk; starting ```

Anyway, I've pushed up a change to the README.me with the new GOROOT-only repro instructions.