bazelbuild / rules_go

Go rules for Bazel
Apache License 2.0
1.37k stars 651 forks source link

go_proto_library: Issue/New Feature for output path generation. #3951

Open abitofhelp opened 4 months ago

abitofhelp commented 4 months ago

Hello, I am unsure whether this is an issue or a new feature request (details follow this introduction). The following describes an issue I encountered with your protobuf plugin when I tried to generate using ConnectRPC's plugin in Bazel 7/bzlmod. Working with the ConnectRPC team, we developed a patch-based solution. I have created a repository for my project with the patch and have donated it to the public domain at https://github.com/abitofhelp/connect-go-example. It would be great if we could add what the patch provides to rules_go. I hope that this information and example code is helpful.

What version of rules_go are you using?

0.47.1

What version of gazelle are you using?

0.36.0

What version of Bazel are you using?

7.2.0

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

MacOS (Sonoma 14.5), AMD64

Any other potentially useful information about your toolchain?

Golang 1.22.3 rules_buf 0.2.0

What did you do?

I created a client/server Go solution that builds with Bazel 7/bzlmod and uses ConnectRPC. I configured the Bazel build to generate the protobuf file without Connect's proto plugin. I configured go_proto_compiler to use Connect's plugin to generate the client/server stubs.

What did you expect to see?

Using Bazel, I generated the protobuf file and it was placed in the correct directory:

Using Bazel, I tried to generate Connect's file in the following location, but nothing was created:

What did you see instead?

Using Bazel, I tried to generate Connect's file and nothing was generated. I contacted Connect's team for assistance debugging the issue.

#########################

ADDITIONAL INFORMATION

######################### Working with ConnectRPC's team, we determined rules_go assumes that generated files go into a single directory (i.e. '/gen/greet/v1'). However, if Connect's generated code is in a separate module from Protobuf's code, it must be placed in a subdirectory named as Connect's generated module (greetv1connect). The expected path for the Connect's generated file would be the following:

In order to support this feature, we created a patch that added a new field to go_proto_library, and it is working properly. The BUILD.bazel file next to the related .proto file has the following go_proto_library blocks. Note the new field: 'overrideimportpath'.

# Generate the greet.connect.pb.go file, which imports greet.pb.go.
go_proto_library(
    name = "greetv1_connect_go_proto",
    compilers = [
        ":connect_go_proto_compiler",
    ],
    importpath = "github.com/abitofhelp/connect-go-example/gen/greet/v1",
    overrideimportpath = "github.com/abitofhelp/connect-go-example/gen/greet/v1/greetv1connect",
    proto = ":greetv1_proto",
    visibility = ["//visibility:public"],
    deps = [":greetv1_go_proto"],
)

# Generate the greet.pb.go file.
go_proto_library(
    name = "greetv1_go_proto",
    importpath = "github.com/abitofhelp/connect-go-example/gen/greet/v1",
    proto = ":greetv1_proto",
    visibility = ["//visibility:public"],
)

FWIW: When I use protoc or 'buf generate proto' rather than bazel to build the solution, they generate the pb and connect files in the proper directory structure.

mickeyh commented 2 months ago

The provided patch seems a bit hacky to upstream. A cleaner solution may be to directly add support for proto plugins that generate output in a subpackage, but it isn't immediately obvious to how to do this.

From what I can tell, the "key" problem seems to be that rules_go constructs expected output paths for proto plugins by appends a source file's basename to the import path. But Connect generates within a subdirectory of the form {proto package}connect. It's not clear to me how to get the {proto package} from inside Starlark though, which makes solutions along these lines annoying.

If Connect is open to it, the most realistic solution would probably be to have protoc-gen-connect-go expose an option to colocate generated code as other proto plugins do. I've posted about this at https://github.com/connectrpc/connect-go/discussions/310#discussioncomment-10016813 but am not sure that the Connect maintainers will be receptive (please upvote that discussion if it would help you!).