Support for automated module fixups for changed dependencies

[ar3s3ru]

The Gazelle plugin at the moment only works on buf_dependencies when running update-repos, which is mostly geared to WORKSPACE-based projects.

There is support to pull in Buf dependencies in a Bzlmod context with the module extensions, but that's mostly manual work (correct me if I'm wrong).

However, Bazel has support for automatic use_repo fixups (source: https://github.com/bazelbuild/bazel/issues/17908), and can be seen in action here for Go: https://github.com/bazelbuild/bazel-gazelle/pull/1511#pullrequestreview-1414973787

We could add something similar here too, so that running bazel run :gazelle can output dependencies fixes, and applied with a script similar to this:

#!/usr/bin/env bash

set -Eeo pipefail

set +e
echo "==> running gazelle through 'bazel run :gazelle'"
buildozer_cmd="$(bazel run :gazelle 2>&1 | grep "buildozer 'use_repo")"
set -Eeo pipefail

if [ -z "$buildozer_cmd" ]; then
  exit 0
fi

echo "==> fixing module dependencies with buildozer"
echo "----> running $buildozer_cmd"
bash -c "$buildozer_cmd"

[srikrsna-buf]

I am not sure about buildozer, need to look into that. But I did want to add support for reading the lock files directly to get the versions similar to how it works in go

[ar3s3ru]

The way it currently works with Gazelle's go is the following:

go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
go_deps.from_file(go_mod = "//:go.mod")
use_repo(
    go_deps,
    "com_connectrpc_connect",
    "com_connectrpc_grpchealth",
    "com_connectrpc_grpcreflect",
    # ...
)

When running bazel run :gazelle, if there is a mismatch between the contents of go.mod and what's in the MODULE.bazel's use_repo, Bazel will output the buildozer commands necessary to run to fix use_repo(go_deps, ...).

Example:

buildozer 'use_repo_remove @gazelle//:extensions.bzl go_deps bazel_gazelle_go_repository_config' //MODULE.bazel:all

In the current dependencies implementation for Bzlmod (very elegant btw 👏🏻), the dependencies are defined with the extension method buf.dependencies(), but are then loaded automatically under @buf_deps.

One feature we could add is something like:

buf_deps = use_extension("@rules_buf//path/to:extensions.bzl", "buf_deps")
buf_deps.from_file(buf_lock = "//path/to:buf_lock")

The big change with this approach however would be to create separate workspaces for each dependency, instead of grouping the modules under a single @buf_deps workspace.

Then we could use something like:

use_repo(
  buf_deps,
  "buf_build_googleapis_googleapis",
  # ...
)

and access them using @buf_build_googleapis_googleapis//....

This might also unlock the ability of doing some dependency-level override, e.g. applying a patch to a specific dependency (I have this specific case atm, where buf.build/googleapis/googleapis has a BUILD.bazel file under google/type with one single proto_library -- not sure if the issue is from Gazelle running in buf_dependencies or from the upstream Buf module)

[aaomidi]

Is there any progress on this? Buf seems to be the last thing I need a workplace file for.

[aaomidi]

It might be that we need https://buf.build/docs/build-systems/bazel updated to explain how to use it in module mode?