bazelbuild / rules_go

Go rules for Bazel
Apache License 2.0
1.37k stars 651 forks source link

Dependency cycle after go_proto_reset_plugin_ #3906

Closed ed-irl closed 6 months ago

ed-irl commented 6 months ago

What version of rules_go are you using?

0.46.0. I'm using bzlmod too, btw.

What version of gazelle are you using?

0.35.0

What version of Bazel are you using?

7.0.2

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

MacOS - M2

Any other potentially useful information about your toolchain?

I'm trying to incorporate a golang app in my monorepo build for the first time. I have no toolchain customizations in place.

What did you do?

Attempting to compile a minimal app (terraform provider) produces a dependency cycle:

✗ bazel build //tools/terraform/providers/irl_gcp:irl_gcp_lib
ERROR: /private/var/tmp/_bazel_ekohlwey/bb77f95dcfbd0c560e2d2a842a7d7b94/external/gazelle~0.35.0~go_deps~org_golang_google_protobuf/cmd/protoc-gen-go/BUILD.bazel:15:10: in go_binary rule @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go: cycle in dependency graph:
    //tools/terraform/providers/irl_gcp:irl_gcp_lib (67c5816ae3f3819410207a631feb665dde2fc8df06f943a95ecf9451d1d04c58)
    //tools/terraform/providers/irl_gcp:irl_gcp_lib (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~com_google_cloud_go_storage//:storage (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_grpc//status:status (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_grpc//internal/status:status (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_grpc//codes:codes (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_grpc//internal:internal (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_grpc//resolver:resolver (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_grpc//credentials:credentials (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~com_github_golang_protobuf//proto:go_default_library (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~com_github_golang_protobuf//proto:proto (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//reflect/protodesc:protodesc (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//types/gofeaturespb:gofeaturespb (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//types/gofeaturespb:gofeaturespb_go_proto (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@rules_go~0.46.0//proto:go_proto (f1bcbe190e6ff65e41bb6d12a4b9057ca28b5904ea6870cc6a6620ffb7603403)
    @@rules_go~0.46.0//proto:go_proto_reset_plugin_ (adce0b6bf0ca5e2db0ca702c59b3f2bf977aa9be776d66b056ba000abc24e214)
.-> @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go_lib (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//compiler/protogen:protogen (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//reflect/protodesc:protodesc (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//types/gofeaturespb:gofeaturespb (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//types/gofeaturespb:gofeaturespb_go_proto (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@rules_go~0.46.0//proto:go_proto (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
|   @@rules_go~0.46.0//proto:go_proto_reset_plugin_ (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
`-- @@gazelle~0.35.0~go_deps~org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go (5f23d8d76e3648a4bb027039ab57c03eddcc5d6b0ef3122e84cdc145921c92d6)
ERROR: Analysis of target '//tools/terraform/providers/irl_gcp:irl_gcp_lib' failed; build aborted
INFO: Elapsed time: 15.805s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

Some relevant portions of my files:

Excerpt from MODULE.bazel:

bazel_dep(name = "rules_go", version = "0.46.0")
bazel_dep(name = "gazelle", version = "0.35.0")

go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
go_deps.from_file(go_mod = "//:go.mod")
use_repo(
    go_deps,
    "com_github_hashicorp_terraform_plugin_sdk_v2",
    "com_google_cloud_go_storage",
    "org_golang_google_api",
)

go.mod:

module abc.xyz/irl/irl

go 1.22.1

require (
    cloud.google.com/go/storage v1.40.0
    github.com/hashicorp/terraform-plugin-sdk/v2 v2.33.0
    google.golang.org/api v0.172.0
)

require (
    cloud.google.com/go v0.112.2 // indirect
    cloud.google.com/go/compute v1.25.1 // indirect
    cloud.google.com/go/compute/metadata v0.2.3 // indirect
    cloud.google.com/go/iam v1.1.7 // indirect
    github.com/agext/levenshtein v1.2.3 // indirect
    github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
    github.com/fatih/color v1.16.0 // indirect
    github.com/felixge/httpsnoop v1.0.4 // indirect
    github.com/go-logr/logr v1.4.1 // indirect
    github.com/go-logr/stdr v1.2.2 // indirect
    github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
    github.com/golang/protobuf v1.5.4 // indirect
    github.com/google/go-cmp v0.6.0 // indirect
    github.com/google/s2a-go v0.1.7 // indirect
    github.com/google/uuid v1.6.0 // indirect
    github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
    github.com/googleapis/gax-go/v2 v2.12.3 // indirect
    github.com/hashicorp/go-cty v1.4.1-0.20200414143053-d3edf31b6320 // indirect
    github.com/hashicorp/go-hclog v1.6.3 // indirect
    github.com/hashicorp/go-uuid v1.0.3 // indirect
    github.com/hashicorp/go-version v1.6.0 // indirect
    github.com/hashicorp/hcl/v2 v2.20.1 // indirect
    github.com/hashicorp/logutils v1.0.0 // indirect
    github.com/hashicorp/terraform-plugin-go v0.22.1 // indirect
    github.com/hashicorp/terraform-plugin-log v0.9.0 // indirect
    github.com/mattn/go-colorable v0.1.13 // indirect
    github.com/mattn/go-isatty v0.0.20 // indirect
    github.com/mitchellh/copystructure v1.2.0 // indirect
    github.com/mitchellh/go-testing-interface v1.14.1 // indirect
    github.com/mitchellh/go-wordwrap v1.0.1 // indirect
    github.com/mitchellh/mapstructure v1.5.0 // indirect
    github.com/mitchellh/reflectwalk v1.0.2 // indirect
    github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
    github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
    github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
    github.com/zclconf/go-cty v1.14.4 // indirect
    go.opencensus.io v0.24.0 // indirect
    go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 // indirect
    go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
    go.opentelemetry.io/otel v1.24.0 // indirect
    go.opentelemetry.io/otel/metric v1.24.0 // indirect
    go.opentelemetry.io/otel/trace v1.24.0 // indirect
    golang.org/x/crypto v0.21.0 // indirect
    golang.org/x/mod v0.16.0 // indirect
    golang.org/x/net v0.22.0 // indirect
    golang.org/x/oauth2 v0.18.0 // indirect
    golang.org/x/sync v0.6.0 // indirect
    golang.org/x/sys v0.18.0 // indirect
    golang.org/x/text v0.14.0 // indirect
    golang.org/x/time v0.5.0 // indirect
    golang.org/x/tools v0.19.0 // indirect
    google.golang.org/appengine v1.6.8 // indirect
    google.golang.org/genproto v0.0.0-20240401170217-c3f982113cda // indirect
    google.golang.org/genproto/googleapis/api v0.0.0-20240401170217-c3f982113cda // indirect
    google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda // indirect
    google.golang.org/grpc v1.62.1 // indirect
    google.golang.org/protobuf v1.33.0 // indirect
)

What did you expect to see?

Project compiles, or a helpful error message.

What did you see instead?

Error message regarding cycle. The source of the cycle appears to be @@rules_go~0.46.0//proto:go_proto_reset_plugin_, but this appears to be generated by rules go. Further guidance on debugging would be helpful.

ed-irl commented 6 months ago

Repro: https://github.com/ed-irl/rules-go-cycle-repro

fmeum commented 6 months ago

This is fixed by https://github.com/bazelbuild/bazel-gazelle/commit/b3cddc07edbede6dd7a5c311aac7ac06660aecf0, which will be part of the next release. You can also apply the directive manually with a go_deps.gazelle_override.

jschaf commented 6 months ago

I upgraded to bazel-gazelle 0.36.0 (includes the linked PR) and hit a different cycle:

    @com_github_envoyproxy_protoc_gen_validate//validate:validate_go_proto (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
.-> @io_bazel_rules_go//proto:go_proto (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
|   @com_github_golang_protobuf//proto:go_default_library (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
|   @com_github_golang_protobuf//proto:proto (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
|   @org_golang_google_protobuf//reflect/protodesc:protodesc (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
|   @org_golang_google_protobuf//types/gofeaturespb:gofeaturespb (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
|   @org_golang_google_protobuf//types/gofeaturespb:gofeaturespb_go_proto (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)
`-- @io_bazel_rules_go//proto:go_proto (04cc90e496cdac7fb6529f87071abff43883905b506a13324c2953baaa3c48d9)

The commonality looks like @org_golang_google_protobuf//types/gofeaturespb:gofeaturespb_go_proto. The package contains generated protobuf code: https://github.com/protocolbuffers/protobuf-go/tree/master/types/gofeaturespb

fmeum commented 6 months ago

@jschaf Is this also fixed by adding a go_deps.gazelle_override with directives = ["gazelle:proto disable"] for that Go module?

jschaf commented 6 months ago

Yes, I'm using a workspaces rather than bzlmod, so the following worked for me:

    go_repository(
        name = "org_golang_google_protobuf",
        build_directives = [
            "gazelle:proto disable",  # https://github.com/bazelbuild/rules_go/issues/3906
        ],
        build_extra_args = [
            "-exclude=**/testdata",
        ],
        importpath = "google.golang.org/protobuf",
        sum = "h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=",
        version = "v1.33.0",
    )
jschaf commented 6 months ago

This broke our CI with gazelle:proto disable. Looks like https://github.com/protocolbuffers/protobuf/issues/16163, so unrelated to this issue. Posting here in case anyone else hits it.

Can confirm, bumping github.com/golang/protobuf to 1.5.4 allows this this change to compile. https://github.com/flipt-io/flipt/pull/2855/files

GoCompilePkg external/com_github_golang_protobuf/protoc-gen-go/descriptor/descriptor.a failed:
(Exit 1): builder failed: error executing command (from target @com_github_golang_protobuf//protoc-gen-go/descriptor:descriptor)

external/go_sdk/builder_reset/builder compilepkg -sdk external/go_sdk -installsuffix darwin_arm64 -src ... (remaining 37 arguments skipped)

external/com_github_golang_protobuf/protoc-gen-go/descriptor/descriptor.pb.go:106:61:
undefined: descriptorpb.Default_FileOptions_PhpGenericServices