Force loading Python rules from rules_python repo

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system

https://bazel.build

Apache License 2.0

23k stars 4.03k forks source link

Force loading Python rules from rules_python repo #8893

Open brandjon opened 5 years ago

brandjon commented 5 years ago

Similar to #8741 and #7643, force users to load bazelbuild/rules_python in order to access the built-in Python rules. This allows us to migrate these rules from native to Starlark without necessarily forcing users to migrate every step of the way. The incompatible change flag will be --incompatible_load_python_rules_from_bzl.

Checklist:

[x] Re-export or copy bundled symbols to bazelbuild/rules_python
[x] Fix rules_python's workflows for precompiled binaries and documentation generation
[x] Rename its canonical workspace to @rules_python
[x] Give the Bazel source tree an internal-only mechanism for replicating rules_python's behavior without loading it
[x] Update all uses of Python in the Bazel source tree (possibly excluding tests and examples) to use the internal mechanism, including the third_party/ tree
[x] Update help and error strings in Bazel to reference @rules_python instead of @bazel_tools
[x] Add the incompatible change flag
[x] Add automated buildifier fixes
[x] Migrate bazelbuild/rules_python and google/subpar for flag flip
[ ] Update Bazel tests for flag flip
[ ] Flip default at head

brandjon commented 5 years ago

The Python rules API consists of the following symbols:

Native rules: py_library, py_binary, py_test, py_runtime
Native providers: PyInfo, PyRuntimeInfo
Starlark-defined rules and aspects: py_runtime_pair, find_requirements
Targets related to selecting on the version: :python_version, :PY2, :PY3
Targets related to defining a toolchain: :toolchain_type, :py2_interpreter_path, py3_interpreter_path
Predefined toolchains: :autodetecting_toolchain, autodetecting_toolchain_nonstrict

The two main goals are to

make these things accessible through rules_python, either by re-exporting them or by copying their definitions; and
make these things inaccessible when rules_python is not used.

In the case of 2, the main priority right now is to prohibit access to the native rules, but not necessarily the other symbols right now, since that would in general require implementing new mechanisms for whitelisting.

brandjon commented 5 years ago

Update: We will be deferring this until Bazel 2.0. In the meantime, projects can and should continue to migrate to using rules_python and not rely directly on bundled Python rules APIs. Note that currently migration is automated with buildifier except for adding the entry to the WORKSPACE file.

abergmeier commented 4 years ago

Soo we are at Bazel 3.0 by now. Any update on when this could happen?

fmeum commented 1 year ago

@lberki The Python runfiles libraries are in a weird state: Their usage instructions recommend to load them from @rules_python, but that repo isn't defined in a default WORKSPACE. If the current issue isn't expected to be resolved soon, https://github.com/bazelbuild/bazel/commit/e4ccba45376f74d24ce3506fb4057e1268f01a59 should probably be reverted.

The actual rules_python repo has its own version of the runfiles libraries forked at a time where it still referred to @bazel_tools: https://github.com/bazelbuild/rules_python/blob/a2b7f4288fc7ad4ed387aa20cb2d09bf497a1b10/python/runfiles/runfiles.py. We should clarify what the source of truth is.