Builds fail with external dependencies managed by `buf.dependency`

[tomasbasham]

Summary

When using buf.dependency dependencies don't seem to be managed correctly which causes a contradiction between the files generated by buf-gazelle and the BUILD.bazel files within the dependencies, resulting in a failed build.

To be honest I'm not entirely sure rules_buf is responsible for this particular issue. I was hoping somebody with more experience could help diagnose the problem.

Bazel Version 7.1.2

To reproduce

Test repo

MODULE.bazel

module(name = "apis")

bazel_dep(name = "rules_buf", version = "0.3.0", dev_dependency = True)
bazel_dep(name = "gazelle", version = "0.34.0", dev_dependency = True)

buf = use_extension("@rules_buf//buf:extensions.bzl", "buf")
buf.toolchains(version = "v1.26.0")

# Allows targets to reference dependencies as @buf_deps.
buf.dependency(module = "buf.build/googleapis/googleapis:f0e53af8f2fc4556b94f482688b57223")
use_repo(buf, "buf_deps")

# repo_name is specified here, using the older name for the protobuf repository,
# because gazelle generates dependencies using the old name.
bazel_dep(name = "protobuf", version = "26.0", repo_name = "com_google_protobuf")

BUILD.bazel

load("@gazelle//:def.bzl", "gazelle", "gazelle_binary")
load("@rules_buf//buf:defs.bzl", "buf_lint_test")
load("@rules_proto//proto:defs.bzl", "proto_library")

# Export the buf.yaml to make it available to rules in other packages.
exports_files(["buf.yaml"])

gazelle(
    name = "gazelle",
    gazelle = ":gazelle-buf",
)

gazelle_binary(
    name = "gazelle-buf",
    languages = [
        "@gazelle//language/proto",
        "@rules_buf//gazelle/buf:buf",
    ],
)

proto_library(
    name = "organisation_service_test_v1_proto",
    srcs = ["test.proto"],
    visibility = ["//visibility:public"],
    deps = [
        "@buf_deps//google/api:api_proto", # here is the contradiction
        "@com_google_protobuf//:field_mask_proto",
    ],
)

buf_lint_test(
    name = "organisation_service_test_v1_proto_lint",
    config = "//:buf.yaml",
    targets = [":organisation_service_test_v1_proto"],
)

test.proto

syntax = "proto3";

package organisation.service.test.v1;

import "google/api/annotations.proto";
import "google/api/client.proto";
import "google/protobuf/field_mask.proto";

service TestService {
  option (google.api.default_host) = "test.organisation.com";

  rpc Get(GetRequest) returns (GetResponse) {
    option (google.api.http) = {
      get: "/v1/{name=workspaces/*/test/*}"
    };
    option (google.api.method_signature) = "name";
  }
}

message GetRequest {
  string workspace = 1;
  string test_id = 2;
  google.protobuf.FieldMask read_mask = 3;
}

message GetResponse {
  Test test = 1;
}

message Test {
  string test_id = 1;
  string name = 2;
}

Expected Behaviour

Running both bazel run //:gazelle and build //:organisation_service_test_v1_proto produces a proto_libray artefact successfully.

Actual Behaviour

The dependencies downloaded and cached from the call to buf.dependency contain a BUILD.bazel file with a single proto_library target given a name equal to the last file in the sources alphabetically.

This contradicts what buf-gazelle expects when generating BUILD files. For example in this case gazelle inserted "@buf_deps//google/api:api_proto" (see above) but this does not exist as demonstrated below.

This of course causes the build to fail.

~/.cache/bazel/_bazel_developer/41361e5cb2e0aa5a08a0816a5f881306/external/rules_buf~~buf~buf_deps/google$ cat api/BUILD.bazel
load("@rules_proto//proto:defs.bzl", "proto_library")

proto_library(
    name = "visibility_proto", # Other side of the contradiction
    srcs = [
        "annotations.proto",
        "client.proto",
        "field_behavior.proto",
        "field_info.proto",
        "http.proto",
        "httpbody.proto",
        "launch_stage.proto",
        "resource.proto",
        "visibility.proto",
    ],
    visibility = ["//visibility:public"],
    deps = [
        "@com_google_protobuf//:any_proto",
        "@com_google_protobuf//:descriptor_proto",
        "@com_google_protobuf//:duration_proto",
    ],
)

[Lumexralph]

@tomasbasham How did you resolve this issue? I seem to be running into this and can't figure out the fix

[tomasbasham]

Hi @Lumexralph. Unfortunately I didn't solve it and resorted to abandoning Buf all together in favour of managing the protobuf dependencies myself. Sorry I could not help.

[Lumexralph]

Hi @Lumexralph. Unfortunately I didn't solve it and resorted to abandoning Buf all together in favour of managing the protobuf dependencies myself. Sorry I could not help.

I was able to solve it eventually using Gazelle resolve directives.

Thanks for your response.