pantsbuild / pants

The Pants Build System
https://www.pantsbuild.org
Apache License 2.0
3.34k stars 638 forks source link

use explicit setup-python versions over implicit expose- (Cherry-pick of #21568, #21582) #21668

Closed huonw closed 1 day ago

huonw commented 3 days ago

The project is currently using a mix of the GitHub supplied setup-python action and pantsbuild/actions/expose-pythons. On GitHub managed runners they behave similarly:

So today the invocation of setup-python followed by expose-pythons is redundant.

Consolidating on setup-python let's us be explicit about expected versions (more like the ARM image) and reduces the number of custom actions the project manages while still making multiple Python versions available.

NOTE: The awkward double newlines in the final yaml are a pre-existing issue https://stackoverflow.com/questions/45004464/yaml-dump-adding-unwanted-newlines-in-multiline-strings

See #21552 for some history regarding the setup- vs expose- actions.

huonw commented 3 days ago

I haven't had time to debug the errors here. Recorded for posterity/convenience:

02:26:58.14 [ERROR] 1 Exception encountered:

Engine traceback:
  in root
    ..
  in pants.core.goals.lint.lint
    `lint` goal

Traceback (most recent call last):
  File "/home/runner/work/pants/pants/src/python/pants/core/goals/lint.py", line 469, in lint
    all_batch_results = await MultiGet(
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 376, in MultiGet
    return await _MultiGet(tuple(__arg0))
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/work/pants/pants/src/python/pants/backend/build_files/fmt/black/register.py", line 21, in black_fmt
    return await _run_black(request, black, black_ics)
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/lint/black/rules.py", line 41, in _run_black
    black_pex, config_files = await MultiGet(black_pex_get, config_files_get)
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 406, in MultiGet
    return await _MultiGet((__arg0, __arg1))
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/work/pants/pants/src/python/pants/backend/build_files/fmt/black/register.py", line 21, in black_fmt
    return await _run_black(request, black, black_ics)
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/lint/black/rules.py", line 41, in _run_black
    black_pex, config_files = await MultiGet(black_pex_get, config_files_get)
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 406, in MultiGet
    return await _MultiGet((__arg0, __arg1))
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/work/pants/pants/src/python/pants/backend/build_files/fmt/black/register.py", line 21, in black_fmt
    return await _run_black(request, black, black_ics)
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/lint/black/rules.py", line 41, in _run_black
    black_pex, config_files = await MultiGet(black_pex_get, config_files_get)
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 406, in MultiGet
    return await _MultiGet((__arg0, __arg1))
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 1085, in create_venv_pex
    venv_pex_result = await Get(BuildPexResult, PexRequest, seeded_venv_request)
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 698, in build_pex
    pex_python_setup, requirements_setup, sources_digest_as_subdir = await concurrently(
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 421, in MultiGet
    return await _MultiGet((__arg0, __arg1, __arg2))
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/work/pants/pants/src/python/pants/engine/rules.py", line 74, in wrapper
    return await call
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 85, in __await__
    result = yield self
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 451, in _determine_pex_python_and_platforms
    python = await Get(
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 365, in find_interpreter
    result = await Get(
  File "/home/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/work/pants/pants/src/python/pants/engine/process.py", line 325, in fallible_to_exec_result_or_raise
    raise ProcessExecutionFailure(
pants.engine.process.ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython<4,>=3.7' failed with exit code 1.
stdout:

stderr:
No supported version of Pip is compatible with the given targets:
cp310-cp310-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.10.15/x64/bin/python3.10
cp311-cp311-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.11.10/x64/bin/python3.11
cp312-cp312-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.12.7/x64/bin/python3.12
cp313-cp313-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.13.0/x64/bin/python3.13
cp37-cp37m-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.7.17/x64/bin/python3.7
cp38-cp38-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.8.18/x64/bin/python3.8
cp38-cp38-manylinux_2_31_x86_64 interpreter at /usr/bin/python3.8
cp39-cp39-manylinux_2_31_x86_64 interpreter at /home/runner/.cache/pants/pants_dev_deps/5c49e358f3370dc4ffcf9529336c30e328019799.venv/bin/python3.9
cp39-cp39-manylinux_2_31_x86_64 interpreter at /opt/hostedtoolcache/Python/3.9.20/x64/bin/python3.9
cburroughs commented 2 days ago

I think the compatible with recent Python pip version is only in Pants 2.24, correct?

huonw commented 2 days ago

Ah yes, not installing Python 3.13 works better.

huonw commented 2 days ago

(Marking this as draft, it doesn't need to be merged before the 2.23.0 release and so better to wait.)