"No repository visible as '@rules_go'" when running gazelle managed by bzlmod

[fishy]

What version of gazelle are you using?

0.37.0

What version of rules_go are you using?

0.48.1

What version of Bazel are you using?

$ bazel version
Bazelisk version: development
INFO: Invocation ID: b30eefbd-ac8e-45f8-a588-36d1cc8fc562
Build label: 7.2.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Tue Jun 25 15:53:05 2024 (1719330785)
Build timestamp: 1719330785
Build timestamp as int: 1719330785

Does this issue reproduce with the latest releases of all the above?

Those are latest releases

What operating system and processor architecture are you using?

linux/amd64

What did you do?

I'm trying to migrate a project using go mod vendor and WORKSPACE managed rules_go and gazelle to bzlmod managed rules_go and gazelle.

I added the following to my MODULE.bazel:

bazel_dep(name = "rules_go", version = "0.48.1")
bazel_dep(name = "gazelle", version = "0.37.0")
# The following 3 were already managed by bzlmod
bazel_dep(name = "rules_oci", version = "1.7.6")
bazel_dep(name = "rules_pkg", version = "0.10.1")
bazel_dep(name = "rules_python", version = "0.33.2")

go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")
go_sdk.download(version = "1.22.5")
go_sdk.host()

go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
go_deps.from_file(go_mod = "//:go.mod")
use_repo(
    go_deps,
    # external deps here omitted
)

# Setup for oci and others omitted

My top level BUILD.bazel file has the following setup for gazelle:

load("@gazelle//:def.bzl", "gazelle")
# other loads

gazelle(name = "gazelle")

# other stuff

But when I try to run bazel run //:gazelle, I'm getting this error:

$ bazel run //:gazelle
INFO: Invocation ID: acb21d7b-efbf-4c5f-a9c5-8c21f1b51349
ERROR: error loading package '@@gazelle~//cmd/gazelle': Unable to find package for @@[unknown repo 'rules_go' requested from @@gazelle~]//go:def.bzl: The repository '@@[unknown repo 'rules_go' requested from @@gazelle~]' could not be resolved: No repository visible as '@rules_go' from repository '@@gazelle~'.
ERROR: /path/to/BUILD.bazel:4:8: every rule of type _gazelle_runner implicitly depends upon the target '@@gazelle~//cmd/gazelle:gazelle', but this target could not be found because of: error loading package '@@gazelle~//cmd/gazelle': Unable to find package for @@[unknown repo 'rules_go' requested from @@gazelle~]//go:def.bzl: The repository '@@[unknown repo 'rules_go' requested from @@gazelle~]' could not be resolved: No repository visible as '@rules_go' from repository '@@gazelle~'.
ERROR: Analysis of target '//:gazelle' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.127s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

I tried to create a minimal reproducible example but cannot reproduce it, so it might be caused by some complexity of my existing project, but I cannot figure out which part went wrong from the error message.

What did you expect to see?

What did you see instead?

[fmeum]

Gazelle refers to rules_go as io_bazel_rules_go in the load statement that fails for you: https://github.com/bazelbuild/bazel-gazelle/blob/852fdcfe7d6cdfcc72b1275a2200a1e131bf0203/cmd/gazelle/BUILD.bazel#L1

Are you sure that you are using upstream Gazelle and don't have any patches or vendoring applied to it? Without a standalone reproducer, I find it more likely that this is an issue specific to your setup (but it's not impossible that there is an actual bug!).

[fishy]

@fmeum Thanks for the reply!

Like I said I tried to reproduce it with a new project with minimal code and failed, so it could totally be that my project did something that breaks it, I just can't find anything that's obviously wrong so hopefully someone more familiar with the code can point me to the right direction 😅 As far as I know I didn't patch gazelle nor rules_go in any way.

I also tried to add repo_name = "io_bazel_rules_go" in bazel_dep in MODULE.bazel but that doesn't make a difference.

The error message also refers to cannot find rules_go not io_bazel_rules_go, which looks weird.

[fmeum]

The error message also refers to cannot find rules_go not io_bazel_rules_go, which looks weird.

Yeah, that's also what confuses me. Could you try looking at the contents of the @gazelle//cmd/gazelle BUILD file under $(bazel info output_base)/external?

[fishy]

You mean this file?

$ cat $(bazel info output_base)/external/gazelle/cmd/gazelle/BUILD.bazel
INFO: Invocation ID: 3ff4b260-ac90-4f5d-b8fe-c179cffc31fc
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
load("//:def.bzl", "DEFAULT_LANGUAGES", "gazelle_binary")

gazelle_binary(
    name = "gazelle",
    languages = DEFAULT_LANGUAGES,
    visibility = ["//visibility:public"],
)

go_library(
    name = "gazelle_lib",
    # keep
    srcs = [
        "diff.go",
        "fix.go",
        "fix-update.go",
        "gazelle.go",
        "metaresolver.go",
        "print.go",
        "profiler.go",
        "update-repos.go",
    ],
    importpath = "github.com/bazelbuild/bazel-gazelle/cmd/gazelle",
    tags = ["manual"],
    visibility = ["//visibility:public"],
    deps = [
        "//config",
        "//flag",
        "//internal/module",
        "//internal/wspace",
        "//label",
        "//language",
        "//language/go",
        "//language/proto",
        "//merger",
        "//repo",
        "//resolve",
        "//rule",
        "//walk",
        "@com_github_pmezard_go_difflib//difflib",
    ],
)

go_test(
    name = "gazelle_test",
    size = "small",
    srcs = [
        "diff_test.go",
        "fix_test.go",
        "integration_test.go",
        "langs.go",  # keep
        "profiler_test.go",
    ],
    args = ["-go_sdk=go_sdk"],
    data = ["@go_sdk//:files"],
    embed = [":gazelle_lib"],
    deps = [
        "//config",
        "//internal/wspace",
        "//testtools",
        "@com_github_google_go_cmp//cmp",
        "@io_bazel_rules_go//go/tools/bazel:go_default_library",
    ],
)

filegroup(
    name = "all_files",
    testonly = True,
    srcs = [
        "BUILD.bazel",
        "diff.go",
        "diff_test.go",
        "fix.go",
        "fix-update.go",
        "fix_test.go",
        "gazelle.go",
        "integration_test.go",
        "langs.go",
        "metaresolver.go",
        "print.go",
        "profiler.go",
        "profiler_test.go",
        "update-repos.go",
    ],
    visibility = ["//visibility:public"],
)

[fishy]

Other things I tried: find . -name "BUILD.bazel" -delete to delete all build files, then repopulate the top level one with only:

load("@gazelle//:def.bzl", "gazelle")

gazelle(name = "gazelle")

And also remove the project .bazelrc file. But I still got the same error.

[fmeum]

Super weird, that file doesn't even contain the failing reference to @rules_go. I have no idea what's here, I fear the only path forward is for you to cut down your example further until it becomes shareable.

[fishy]

this seems to be fixed in gazelle v0.38.0. now that gazelle itself works to update the build files, but I'm hitting some issues with proto dependencies on the third party libraries I use, which is unrelated to this original issue.

[fishy]

Thinking about this issue more, I start to suspect that it was caused by something wrongly cached by bazel.

When I could reproduce it, I was trying to migrate a project from having rules_go and gazelle in WORKSPACE (with old names) to bzlmod (with new names), I also run a bazel remote cache locally (https://github.com/buchgr/bazel-remote run with podman). so when it complains about "cannot find rules_go", it's probably because bazel still picked the old stuff that's under io_bazel_rules_go from the cache, and didn't invalidate that cache correctly? when I try to create a reproducible example, that's a new project starting with bzlmod directly so it won't have any old thing cached to cause the issue.

@fmeum do you think that's plausible?

[fmeum]

It's still weird since build files aren't cached in that kind of cache. But I don't think it's worth investigating further unless it shows up again.

[fishy]

one of my colleague somehow reproduced a very similar issue on our CI system, the error is the "opposite" that it complains about unable to find io_bazel_rules_go:

Unable to find package for @@[unknown repo 'io_bazel_rules_go' requested from @@]//go:def.bzl: The repository '@@[unknown repo 'io_bazel_rules_go' requested from @@]' could not be resolved: No repository visible as '@io_bazel_rules_go' from main repository.

the interesting thing is that what they are doing is that they have a PR from before we migrated from io_bazel_rules_go under WORKSPACE to rules_go under MODULE.bazel, then after we made the bzlmod migration, they merged back the bzlmod migration to their PR, and start to see this issue in CI.

which seem to support my caching issue hypothesis? that because it used to run under the name io_bazel_rules_go, and now it's supposed to be rules_go (as per MODULE.bazel), but bazel still picked up some old stuff from the cache that's under io_bazel_rules_go name?

Nevermind, this particular case turned out to be a false negative. It was caused by bad merge (BUILD.bazel files generated by gazelle while we were still using io_bazel_rules_go name, and not updated to rules_go after merging the bzlmod change).

[fmeum]

It's going to be hard to debug this without a reproducing sample. The interaction between WORKSPACE and MODULE.bazel is subtle enough that I won't rule this out, but if nothing in WORKSPACE (or WORKSPACE.bzlmod) brings in rules_go (as either rules_go or io_bazel_rules_go), this shouldn't happen.