Switch bazel dependency management to bzlmod

[phlax]

Im in no hurry to do this as afaict its likely to break quite a lot of things

That said my understanding is that bzlmod fixes many of the underlying issues in the workspace system, and is the future of bazel dep management, so we probs should embrace sooner or later

[phlax]

cc @keith @htuch @moderation

[moderation]

Looks interesting. Some links for future reference

• Manage external dependencies with Bzlmod
• Bzlmod Migration Guide

[keith]

I think one major thing to figure out with this is how / if it's possible to keep the current REPOSITORY_LOCATIONS_SPEC format. For projects that are generally applicable in the bazel community, like the rulesets or very common C++ libraries, the ideal would be to have a bazel_dep directly in the MODULE.bazel file in the root of the repo. There isn't any flexibility in that format unless you opt-out of the automatic dependency resolution features (which is the primary benefit IMO). Theoretically we could use a separate tool to generate the MODULE.bazel from some source file, but that would add extra complexity to updating dependencies.

[keith]

Another thing we likely need to figure out is removing all uses of bind which is not supported with bzlmod

[keith]

The incremental migration story is pretty good though so if we make decisions on some of these things we could likely get started and try it out. I guess the other concern is how downstream integrators who pull envoy into a bazel build are affected, I think we could come up with a strategy to not break them (for now)

[keith]

Here's an example of how we could get rid of bind, RFC: https://github.com/envoyproxy/envoy/pull/30458

[phlax]

following up offline chat with @keith - probably we can add a dependency-metadata.yaml file or similar to overlay any additional data for our bzlmod deps - this could be used/validated by the dep checkers without any need to dynamically generate module files etc

[keith]

In general having poked around a bit more this is definitely going to be a lot of work. Ideally all the dependencies down the tree would be bzlmod-ified, and that's a lot of changes that need to be made

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

[moderation]

Bazel 7.0 is out with integrated bzlmod - https://blog.bazel.build/2023/12/11/bazel-7-release.html

[phlax]

from my reading of bazel roadmap there is a transitional period - i think version 7.x in which both are supported and WORKSPACE is the default, then the default is flipped, and finally WORKSPACE is removed (8 or 9 iirc)

@keith one thing im increasingly seeing is repos with both systems concurrently - probs that is a good idea for us - we could get started with it experimentally without having to unbreak the current mould

[albertocavalcante]

Is this something you're looking for external contribution? I haven't looked yet at Envoy BUILD files, I bet they're complex, but would like to give it a try if it's ok. Thanks.

[phlax]

hi @albertocavalcante any help with this would be greatly appreciated - my guess is that it will be more of a transition than an immediate change

the bazel dependencies are defined/registered (mostly) in bazel/repository_locations.bzl and bazel/repositories.bzl - there are a few complexities, but shouldnt be so hard to grok

[phlax]

i have added a milestone for this https://github.com/envoyproxy/envoy/milestone/106

@mmorel-35 has been working on this in #32669 and in other envoy repos

[phlax]

@mmorel-35 in terms of transition -> bzlmod for envoy itself - i think we probably need to set a roadmap and announce that to downstreams

the first step is definitely to get something work that can build.

im a bit concerned that we dont have to maintain both systems for longer than necessary - mostly because im guessing updates will need to happen in 2 places etc, and we dont want to have to run all of our ci both ways on every run

so i guess once we have something that reasonably works we should set a date/version for switchover and make that our default at that point (irrespective of the bazel default at that point)

[albertocavalcante]

@phlax would happy to contribute on this one, I'm just checking if I'll be able to contribute as part of my work, hoping for a yes.

[phlax]

sounds good @albertocavalcante - best to coordinate with @mmorel-35 who has done a lot of the ground work for the transition

[aiuto]

Is there any progress on this? Even having a MODULE.bazel that was small with a large WORKSPACE.bzlmod is a great start.

[phlax]

Some progress but only with our self-managed deps - not in repo here - PRs would be welcome