Open UebelAndre opened 7 months ago
I think that something related to this got fixed in bzlmod
by us starting to use is_python_x.y
instead of is_python_x.y.z
config_settings in the hub repos. That might be still present in the WORKSPACE
multi-version plumbing.
I wonder if there are other things that need fixing here as well.
If you explicitly register a 3.11.4
version it will work. But because you haven't, 3.11.4
does not match any registered toolchains and the default version is selected (3.11.6
, since that's the latest in the 3.11
series in rules_python 0.29).
My expectation is that if I register 3.11
, I would get all versions of py311 available registered as toolchains and gated by target_settings
where each toolchain is constrained by the full version (e.g. 3.11.4
) and for only major+minor versions (e.g. 3.11
) an extra toolchain would be registered only for the latest available version with this as a target_setting
. That way if I specify in my bazelrc that I want 3.11.4, I would correctly match it. I would rather not need to specify full versions when registering toolchains in hopes that I would continue to have all versions available to me and let either bazelrc flags and transitions decide what python is desired.
Right now the behaviour is that you get the latest. With bzlmod
, you should be able to specify it in the .bazelrc
and get what you would expect, I think.
I think you would still need to register the desired toolchain version in MODULE.bazel
before selecting it in .bazelrc
.
We could do away with having to explicitly ask for the versions by just always registering everything. That should be relatively cheap because bzlmod won't actually instantiate those repos unless they're actually used.
It's too bad flags can't be used in bzlmod or repo phases. The closest we could get is an environment variable to control things, which is awkward. Could be useful as an optimization to limit how many unnecessary versions get registered.
Maybe a happy medium is to register 3.11.* if someone asks for 3.11
? Otherwise the list of registered repos grows forever.
Or maybe that's what you're suggesting?
Yeah. Or maybe always register the latest version of each?
Something I've been seeing library modules doing is using list comps in their MODULE files to request every version available. They don't know which version will ultimately be used, so they just have to list every version. Which is annoying for them. Plus when a new version is added, they'll have to update again. So double annoying.
I agree that we should probably just register the latest versions by default.
Experiencing python 3.11.1 being selected by default for PY3 with bazel 7.3.1 and rules_python 0.35.0
relevant output with --toolchain_resolution_debug
(19:47:42) INFO: ToolchainResolution: Performing resolution of @@bazel_tools//tools/python:toolchain_type for target platform @@platforms//host:host
ToolchainResolution: Rejected toolchain @@rules_python~~python~python_3_11_aarch64-apple-darwin//:python_runtimes; mismatching values: osx, aarch64
ToolchainResolution: Rejected toolchain @@rules_python~~python~python_3_11_aarch64-unknown-linux-gnu//:python_runtimes; mismatching values: aarch64
ToolchainResolution: Rejected toolchain @@[unknown repo 'python_3_11_ppc64le-unknown-linux-gnu' requested from @@rules_python~~python~pythons_hub]//:python_runtimes; mismatching values: ppc
ToolchainResolution: Rejected toolchain @@rules_python~~python~python_3_11_x86_64-apple-darwin//:python_runtimes; mismatching values: osx
ToolchainResolution: Rejected toolchain @@rules_python~~python~python_3_11_x86_64-pc-windows-msvc//:python_runtimes; mismatching values: windows
ToolchainResolution: Toolchain @@rules_python~~python~python_3_11_x86_64-unknown-linux-gnu//:python_runtimes is compatible with target plaform, searching for execution platforms:
ToolchainResolution: Compatible execution platform @@platforms//host:host
ToolchainResolution: All execution platforms have been assigned a @@bazel_tools//tools/python:toolchain_type toolchain, stopping
ToolchainResolution: Recap of selected @@bazel_tools//tools/python:toolchain_type toolchains for target platform @@platforms//host:host:
ToolchainResolution: Selected @@rules_python~~python~python_3_11_x86_64-unknown-linux-gnu//:python_runtimes to run on execution platform @@platforms//host:host
haven't yet quickly found why that 3.11 ends up being 3.11.1 instead of 3.11.9
Motivation is that 3.11 until 3.11.9 dynamically links against libcrypt.so.1
which not all systems have nowadays
Haven't been able to pin it to 3.11.9 neither with python_register_multi_toolchains
nor --@rules_python//python/config_settings:python_version=3.11.9
or a combination thereof
$ bazel mod show_repo @@rules_python~~python~python_3_11_x86_64-unknown-linux-gnu
(20:03:45) INFO: Current date is 2024-09-12
## @@rules_python~~python~python_3_11_x86_64-unknown-linux-gnu:
# <builtin>
python_repository(
name = "rules_python~~python~python_3_11_x86_64-unknown-linux-gnu",
distutils_content = "",
ignore_root_user_error = False,
patches = [],
platform = "x86_64-unknown-linux-gnu",
python_version = "3.11.1",
release_filename = "20230116/cpython-3.11.1+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz",
sha256 = "02a551fefab3750effd0e156c25446547c238688a32fabde2995c941c03a6423",
strip_prefix = "python",
urls = ["https://github.com/indygreg/python-build-standalone/releases/download/20230116/cpython-3.11.1+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz"],
)
Hmm, consuming via WORKSPACE selects 3.11.1, but consuming via MODULE.bazel sleects 3.11.9 :thinking:
π bug report
Affected Rule
The issue is caused by the rule: `python_register_multi_toolchains` ### Is this a regression?I think this has always been a bug.
Description
A clear and concise description of the problem... `@rules_python//python/config_settings:python_version` does not control the of the python toolchain that's used when paired with `python_register_multi_toolchains` ## π¬ Minimal ReproductionUsing a simple WORKSPACE like the one below, I'm able to invoke the following to see the version of python printed does not match the version specified in the command line
WORKSPACE.bazel
BUILD.bazel
example.py
π₯ Exception or Error
The example above will print
py3.11.6
, the newest available toolchain, but not the one specified on the command line.π Your Environment
Operating System:
Output of
bazel version
:Rules_python version:
Anything else relevant?