bazelbuild / rules_python

Bazel Python Rules
https://rules-python.readthedocs.io
Apache License 2.0
541 stars 543 forks source link

Missed incrementing module compatibility_level after changing pip_parse module extension repo name tag to `hub_name` #2459

Open mikn opened 1 day ago

mikn commented 1 day ago

Affected Rule

https://registry.bazel.build/modules/grpc (probably more)

The issue appears in the pip module extension (@rules_python//python/extensions:pip.bzl), specifically renaming the name tag to hub_name without incrementing the rules_python module compatibility level from 1 to 2.

Is this a regression?

Yes - rules_python didn't increment its compatibility_level when introducing a breaking change from name to hub_name in the pip extension, causing issues with transitive dependencies that use the older API.

Description

The core issue is that rules_python made a breaking API change (changing name to hub_name in the pip extension) without incrementing the compatibility_level parameter from 1 (set in version 0.22) to 2. This causes problems in dependency chains where:

  1. A root module uses rules_python 1.0.0-rc2 (which requires hub_name)
  2. A transitive dependency (in this case, protoc-gen-validate via grpc) uses an older version with name
  3. Because the compatibility_level wasn't incremented, Bazel doesn't recognize this as a breaking change and attempts to resolve both versions together

This manifests as an error when running bazel mod tidy:

ERROR: in tag at https://bcr.bazel.build/modules/protoc-gen-validate/1.0.4/MODULE.bazel:64:10, unknown attribute name provided. Type 'bazel help mod' for syntax and help.

🔬 Minimal Reproduction

Here's a minimal reproduction case demonstrating the compatibility level issue:

  1. MODULE.bazel:
    
    bazel_dep(name = "rules_python", version = "1.0.0-rc2")
    bazel_dep(name = "protobuf", version = "29.0")
    bazel_dep(name = "grpc", version = "1.68.0")

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( hub_name = "pypi", # New API python_version = "3.11", requirements_lock = "//python:requirements.txt", ) use_repo(pip, "pypi")


The issue occurs because protoc-gen-validate (introduced as a transitive dependency by grpc) uses the older `name` parameter, but the compatibility_level wasn't incremented to prevent this version mixing.

The complete reproduction case includes additional files (BUILD.bazel, proto definitions, and Python code) which can be found in the linked BCR issue: https://github.com/bazelbuild/bazel-central-registry/issues/3291

## 🔥 Exception or Error

bazel mod tidy ERROR: in tag at https://bcr.bazel.build/modules/protoc-gen-validate/1.0.4/MODULE.bazel:64:10, unknown attribute name provided. Type 'bazel help mod' for syntax and help.


## 🌍 Your Environment
**Operating System:**

Linux (amd64)


**Output of `bazel version`:**

bazel 7.4.1


**Rules_python version:**

1.0.0-rc2


**Anything else relevant?**

Current workaround is to use `multiple_version_override`:
```python
multiple_version_override(
    module_name = "rules_python",
    versions = [
        "1.0.0-rc2",
        "0.22.0",
    ],
)

The proper fix would be to increment the compatibility_level parameter in rules_python. This would prevent Bazel from attempting to mix incompatible versions of the rules in a single build.

aignas commented 1 day ago

I do remember a discussion somewhere in the early days of bzlmod that the compatibility_level bumping is super painful but I definitely don't have a reference anymore. @Wyverald, do you have any recommendation how rules_python should resolve this?

e.g. is a retroactive backfill of the bumped compatibility_level after we release a new version with a bumped level would be something that cannot be done? I assume we might be able to pull it off by applying extra patches to the MODULE.bazel metadata in the registry.