Closed choo-stripe closed 3 years ago
Hey @choo-stripe,
I would like to have this issue sit a while for discussion from the community. I know there are other bazel users today and mockgen works for them from what I have heard. I assume this is because their build environment has Go in it. In general I don't think having Go in the environment is a bad thing. mockgen is meant to be a tool to generate mocks so you can write tests against. In order to run those test you will also need Go.
Hi @codyoss,
Thanks for the response! For context, the issue is not with the go
tool itself, but with the dependency on a $GOPATH
:
$GOPATH
-style directory, containing the full transitive dependencies of a library. This directory is expensive to create as part of a Bazel build action.$GOPATH
.$GOPATH
mechanism.Why can you not specify with GOPATH where your sources are? I feel like this could work?
@linzhp I know you have talked about using bazel and mockgen in the past. Do you have thoughts here?
@codyoss Thanks for tagging me. Yes! I have some thoughts.
- Some auxiliary Go tooling assumes the presence of a $GOPATH-style directory, containing the full transitive dependencies of a library. This directory is expensive to create as part of a Bazel build action.
- I don't have full context on this, but it looks like mockgen is using a library to detect the package name to generate code for, and that library depends on $GOPATH.
I can give some context here. Before #420, mockgen used golang.org/x/tools/go/packages.Load
, which calls go list
, to determine the the package path given a source file. go list
requires a GOPATH
with a full transitive closure of dependencies, which is very I/O intensive to populate from scratch. When running outside Bazel, there is already either a full GOPATH
or mod cache available, so mockgen doesn't have to populate it every time. However, they are not available in Bazel's sandbox for hermeticity, so they had to be populated for every target that calls mockgen in source mode, making them among the slowest targets to build in Uber's Go monorepo.
GOPATH
by guessing import paths from the source file's relative path in GOPATH
. So Bazel only needs to populate a small GOPATH
with the source file and aux files in it, instead of the full transitive closure. This greatly improves the build performance. Although we have been benefiting from such improvement internally at Uber, our pull request got stuck due to backward compatibility concerns and inactivity of the upstream project too.I don't think this proposal can significantly improve the Bazel build performance over the combination of #420 and https://github.com/jmhodges/bazel_gomock/pull/44. However, it would make mockgen more flexible, further breaking the tie to GOPATH
, which is going away in Go 1.17. It also makes build systems like Bazel easier to call mockgen, because they no longer need to put the source file in a structure to resemble GOPATH
. Such structure is often not readily available in their runtime environment, requiring extra copying.
So I see this as a usability improvement, with the cost of supporting an extra flag in mockgen.
To the point of
My end goal is to write a bazel rule that can mockgen for grpc clients.
You can do this without writing a Bazel rule on your own. For example:
load("@rules_proto//proto:defs.bzl", "proto_library")
load("@com_github_jmhodges_bazel_gomock//:gomock.bzl", "gomock")
load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
load("@io_bazel_rules_go//go:def.bzl", "go_library")
proto_library(
name = "paxoshpb_proto",
srcs = ["paxosh.proto"],
visibility = ["//visibility:public"],
)
go_proto_library(
name = "paxoshpb_go_proto",
compilers = [
"@io_bazel_rules_go//proto:gogoslick_grpc",
],
importpath = "proto.example.com/paxosh",
proto = ":paxoshpb_proto",
visibility = ["//visibility:public"],
)
go_library(
name = "go_default_library",
embed = [":paxoshpb_go_proto"],
importpath = "proto.example.com/paxosh",
visibility = ["//visibility:public"],
)
gomock(
name = "mocks",
out = "mocks.go",
interfaces = [
"PaxosHClient",
"PaxosHServer",
],
library = ":go_default_library",
package = "paxosh",
)
go_library(
name = "go_mocks_library",
srcs = ["mocks.go"],
importpath = "mock.proto.example.com/paxosh",
visibility = ["//visibility:public"],
deps = [
":go_default_library",
"@com_github_golang_mock//gomock:go_default_library",
"@org_golang_google_grpc//:go_default_library",
],
)
Note that the example use reflective mode of mockgen, because the generated source file in go_proto_library
is not visible
Requested feature A clear description of the desired feature and an example of how it would be used.
There should be a way to specify the source-package such that mockgens requiring structs from the source file can succeed outside of a standard go environment.
For example, I should be able to pull a mockgen binary, and in a fresh docker container run:
Today, that'd fail.
Why the feature is needed A clear description of how this feature is not served by existing functionality in gomock.
My end goal is to write a bazel rule that can mockgen for grpc clients. The great part about this is - as the proto schemas change, bazel will notice the dependency changed and re-run mockgen, ensuring the mock and source stay in-sync.
Currently
mockgen
uses standardgo
tooling to find the package of the source file submitted - go modules or poking around inGOPATH
. This makes it incompatible with build systems like bazel, which run their own go toolchain to compile go files. Bazel executes builds in isolated build environments, which don't necessarily have access to go modules or GOPATH.Now,
mockgen
is a tool. While it does go-like things i.e. codegen, the binary should be portable and able to execute outside of a go environment. The problem, obviously, is that mockgen can't really know the full package path for some arbitrary source file you give it, hence it relies on the go env.So! If only we could override the package where a source file comes from, it wouldn't require a full-blown go env setup (also the behaviour is a bit too magical right now), and we'd be able to portably call
mockgen -source=foo.go -source_package=my/foo
, such that the generated mock would properly import frommy/foo
.(Optional) Proposed solution A clear description of a proposed method for adding this feature to gomock.
I wrote up a PR https://github.com/golang/mock/pull/529