go_repository build_file_proto_mode - want proto_library + go_library but not go_proto_library

ash2k commented 4 years ago

What version of gazelle are you using?

v0.22.2

What version of rules_go are you using?

v0.24.3

What version of Bazel are you using?

v3.6.0

Does this issue reproduce with the latest releases of all the above?

Yes.

What operating system and processor architecture are you using?

macOS x86_64

Issue description

I'm trying to use https://github.com/envoyproxy/protoc-gen-validate to generate validation code for my protobufs. I'm importing it like that:

go_repository(
    name = "com_github_envoyproxy_protoc_gen_validate",
    build_file_proto_mode = "disable_global",
    build_naming_convention = "go_default_library",
    importpath = "github.com/envoyproxy/protoc-gen-validate",
    sum = "h1:A9nAQ7H0O/o654GnqyDZtNAdbvXIl5hf+OsYAzBfDx0=",
    version = "v0.4.2-0.20200930220426-ec9cd95372b9",
)

I'm getting:

package conflict error: github.com/golang/protobuf/ptypes/any: multiple copies of package passed to linker:
        @com_github_golang_protobuf//ptypes/any:any
        @io_bazel_rules_go//proto/wkt:any_go_proto
Set "importmap" to different paths or use 'bazel cquery' to ensure only one
package with this path is linked.
This will be an error in the future.

This is because this repository uses option 1 (go_proto_library) and I'm using options 2 (pre-generated files) from https://github.com/bazelbuild/rules_go/blob/master/proto/core.rst#avoiding-conflicts.

To generate Go files (so that I can check them in) from my proto files AND the proto file in this library I need the proto_library for the file from the library. But I also need to have go_library for the pre-generated Go file from this library. I don't see how I can achieve that today using the build_file_proto_mode parameter.

I'd like to suggest to introduce a new mode that addresses this use case. It should generate proto_library and go_library targets for the go_repository.

Workarounds that I know of:

Maintain a patch that go_repository can use to alter the existing build files in the repository.
Somehow only use the .proto file from the library but not the other targets. Not sure if this will even work and it's very ugly.

See this PR for full example https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/132/diffs

p.s. also it'd be nice if the error message told me what cquery command to run.

jayconrod commented 4 years ago

This repository already has build files, so by default, go_repository won't run Gazelle. Even with build_file_generation = "on", I don't think Gazelle will work reliably in a situation like this.

For repos that already have build files, I'd generally recommend http_archive with patches instead of go_repository.

ash2k commented 4 years ago

@jayconrod Thanks for your reply. I've managed to get it working with a go_repository + a patch, it was not a big deal. However, it only solved the warnings for my own packages. I still get the same warning for the protoc plugin that this repository provides. Interestingly, if I clone the repo and run the build, there are no warnings. Not sure why that happens and I ran out of time budget to debug this.

Regarding the proposal. I see your point. Maybe another option would make sense then - fully ignore existing build files? I've had a similar situation before where I needed to use a repository with existing build files. I was lucky because they were called BUILD so I told Gazelle to generate BUILD.bazel, which take priority and the existing files were effectively ignored. With the current project the same approach does not work because they have a mix of both BUILD and BUILD.bazel. If go_repository allowed to ignore them all then I'd be able to tell it how and what to generate and apply a tiny corrective patch on top to just add proto_library that I need (without go_proto_library). WDYT?

rytswd commented 3 years ago

Sorry to jump in, but I'd love to see this capability of ignoring BUILD and BUILD.bazel files from the destination as well. I was hit by the existing BUILD.bazel causing build failure as detailed in https://github.com/bazelbuild/rules_go/issues/2738, and I ended up seeking for further help from the destination project (ref: https://github.com/grpc-ecosystem/grpc-gateway/issues/1847). If I could ignore their BUILD files, I think that would allow me to build without any change on their side.

ash2k commented 3 years ago

Perhaps this can be helpful to someone who's trying to fix similar issues. I was able to fix the upstream dependency (https://github.com/envoyproxy/protoc-gen-validate/pull/413) and use it (https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/206). rules_go v0.25.0 forced me to look into this and find a working solution 🤷

bazel-contrib / bazel-gazelle