Open chandraaditya opened 10 months ago
@tyler-french What do you think, are there any footguns in allowing multiple go.mod
files per module? Proper go.work
support would of course be more involved.
@tyler-french What do you think, are there any footguns in allowing multiple
go.mod
files per module? Propergo.work
support would of course be more involved.
I want to better understand the reason behind having the multiple go.mod
files.
Are the different projects built with different resolved versions of the different modules? In this case, supporting multiple go.mod
files would not have the desired effect, as they would deterministically resolve to a single set of dependencies.
Are there any exclude
or replace
directives in these go.mod
files? If so, allowing multiple go.mod
files is actually a bit dangerous, because it might make the resolved versions from MVS non-deterministic (or dependent on the ordering of tags)
I'm wondering if it might be more practical to have one MODULE.bazel
per go.mod
file, each as a separate "module", and then you could have a root module /MODULE.bazel
that imports the others (this should work, right @fmeum?), does resolution, and applies overrides. This would also be more "correct", in terms of following the duality of a go.mod
module and a Bazel module.
I'm wondering if it might be more practical to have one MODULE.bazel per go.mod file, each as a separate "module", and then you could have a root module /MODULE.bazel that imports the others (this should work, right @fmeum?), does resolution, and applies overrides. This would also be more "correct", in terms of following the duality of a go.mod module and a Bazel module.
Actually I think yes, this would be a better question to ask as there's no good documentation on the best practices with gazelle and Bzlmod anywhere(specifically when it comes to monorepos).
Hello,
I am currently using a monorepo with some multiple go modules inside, managed via bzlmod.
I can share my approach, not optimal but at least it seems working..
see here: https://github.com/opicaud/monorepo
FYI I don't use any exclude
or replace
directives, but i must duplicate a lot of configuration, specifically extensions go_deps
and directive bazel_deps
I have also those kind of warnings, which are "normal" root module are not using any kind of go modules..
WARNING: /__w/monorepo/monorepo/MODULE.bazel:112:24: The module extension go_deps defined in @bazel_gazelle//:extensions.bzl reported incorrect imports of repositories via use_repo():
Imported, but reported as indirect dependencies by the extension:
com_github_beorn7_floats, com_github_cucumber_godog, com_github_davecgh_go_spew, com_github_google_uuid, com_github_grpc_ecosystem_grpc_gateway_v2, com_github_hashicorp_logutils, com_github_pact_foundation_pact_go_v2, com_github_smarty_assertions, com_github_spf13_pflag, com_github_spf13_viper, com_github_stretchr_testify, com_github_urfave_cli_v3, org_golang_google_grpc, org_golang_x_exp, org_golang_x_mod
** You can use the following buildozer command to fix these issues:
buildozer 'use_repo_remove @bazel_gazelle//:extensions.bzl go_deps com_github_beorn7_floats com_github_cucumber_godog com_github_davecgh_go_spew com_github_google_uuid com_github_grpc_ecosystem_grpc_gateway_v2 com_github_hashicorp_logutils com_github_pact_foundation_pact_go_v2 com_github_smarty_assertions com_github_spf13_pflag com_github_spf13_viper com_github_stretchr_testify com_github_urfave_cli_v3 org_golang_google_grpc org_golang_x_exp org_golang_x_mod' //MODULE.bazel:all
Hope it can help
I'm also trying to understand how rules_go
is supposed to work with bzlmod + multiple interconnected go modules and I ended up here.
(as well as relatedly https://github.com/bazelbuild/bazel-gazelle/issues/1715)
Concretely -
MODULE.bazel
at my repository root alongside WORKSPACE
, or many MODULE.bazel
files, one per go.mod
, and perhaps need to introduce tooling to interconnect them. @fmeum says "one" here: https://github.com/bazelbuild/bazel-gazelle/issues/1715#issuecomment-1878666270 and @tyler-french says "many" here: https://github.com/bazelbuild/bazel-gazelle/issues/1658#issuecomment-1795201111. Or maybe it is supposed to be only one MODULE.bazel
+ one go.mod
for a whole monorepo?Here is a setup that is not pleasant but it works (without bzlmod). I can't see how to port it to bzlmod.
# gazelle:repository_macro go_repositories.bzl%go_repositories
go_repositories()`
go.mod
file and a BUILD.bazel
file. To add an external dep, run go get
(to get transitive dependencies), then bazel run //:gazelle
to move those to BUILD.bazel
.replace
statements to point to each other via their paths in the monorepo, to allow go get
to work.github.com/airyhq/airy/tools/update-deps
tool to sync all go.mod
files to a consistent, repo-wide version.gazelle:prefix
and gazelle:proto_strip_import_prefix
fixes to manage the names/go module paths of shared .proto
files that aren't in the same part of the tree as most of the go code.A true monorepo would only have a single MODULE.bazel
and a single go.mod
file. If you would like to still have multiple go.mod
files, you might be interested in go.work
support (see https://github.com/bazelbuild/bazel-gazelle/pull/1731).
Replace directives mentioning local paths are waiting for Bazel features that would allow us to overlay a local repo with generated build files. It's also actively being worked on.
My current state is multiple go.mod
files with no go.work
. I'm experimenting with cutting down to one go.mod
file for the whole repo based on your suggestion. Are you aware of any published examples of multi-language monorepos that follow this design? It would help me a lot, and I'm sure others. I think it would be very helpful to have some expert documentation or example about the recommended workflows.
To satisfy both go mod
and gazelle
, it looks like I need to
go.mod
. Does that sound right? The protobuf shadow directory makes me suspicious I'm still missing some best practices.
This PR may be relevant: https://github.com/bazelbuild/bazel-gazelle/pull/1731
What version of gazelle are you using?
v0.33.0
What version of rules_go are you using?
v0.41.0
What version of Bazel are you using?
7.0.0-pre.20231011.2
Does this issue reproduce with the latest releases of all the above?
Yes
What operating system and processor architecture are you using?
macos 14.1 Beta (23B5056e), x86
What did you do?
I'm basically trying to move to Bzlmod from WORKSPACE, and I'm confused about how to use gazelle with it in a monorepo.
The docs at https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md say that gazelle will use go_deps, and that I don't need to run
gazelle update-repos
anymore but the problem is that I can only define onego_deps.from_file
in my MODULE.bazel file, but I have multiple go projects in my monorepo with different dependencies for each.With the WORKSPACE file it was at least possible to run
update-repos
for each go.mod file, but I do know that that's not the recommended way of doing it, and that gazelle has a tool to combine all go.mod files into a single go.mod file in the repo's root.But is there a way to still use multiple go.mod files?