actions / setup-python

Set up your GitHub Actions workflow with a specific version of Python
MIT License
1.59k stars 505 forks source link

`setup-python` failing for 3.8 on `macos-latest` #860

Open abravalheri opened 1 month ago

abravalheri commented 1 month ago

Description: The setup-python action fails when installing Python 3.8 on the macos-latest. Please note that the setup-python action is the second one to run, just after checkout.

https://github.com/pypa/setuptools/actions/runs/8982558570/job/24670435133?pr=4329#step:3:32

```python 2024-05-07T08:58:00.5927920Z Current runner version: '2.316.1' 2024-05-07T08:58:00.5947060Z ##[group]Operating System 2024-05-07T08:58:00.5947530Z macOS 2024-05-07T08:58:00.5947810Z 14.4.1 2024-05-07T08:58:00.5948080Z 23E224 2024-05-07T08:58:00.5948370Z ##[endgroup] 2024-05-07T08:58:00.5948680Z ##[group]Runner Image 2024-05-07T08:58:00.5949020Z Image: macos-14-arm64 2024-05-07T08:58:00.5949360Z Version: 20240422.3 2024-05-07T08:58:00.5950120Z Included Software: https://github.com/actions/runner-images/blob/macos-14-arm64/20240422.3/images/macos/macos-14-arm64-Readme.md 2024-05-07T08:58:00.5951200Z Image Release: https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20240422.3 2024-05-07T08:58:00.5951860Z ##[endgroup] 2024-05-07T08:58:00.5952180Z ##[group]Runner Image Provisioner 2024-05-07T08:58:00.5952640Z 2.0.369.1+55cddbb57f254a369b6e7b3b508cebdbee5d24be 2024-05-07T08:58:00.5953080Z ##[endgroup] 2024-05-07T08:58:00.5953760Z ##[group]GITHUB_TOKEN Permissions 2024-05-07T08:58:00.5954710Z Contents: read 2024-05-07T08:58:00.5955040Z Metadata: read 2024-05-07T08:58:00.5955360Z ##[endgroup] 2024-05-07T08:58:00.5957230Z Secret source: None 2024-05-07T08:58:00.5957630Z Prepare workflow directory 2024-05-07T08:58:00.6611680Z Prepare all required actions 2024-05-07T08:58:00.6719890Z Getting action download info 2024-05-07T08:58:00.9287680Z Download action repository 'actions/checkout@v4' (SHA:0ad4b8fadaa221de15dcec353f45205ec38ea70b) 2024-05-07T08:58:01.9736320Z Download action repository 'actions/setup-python@v5' (SHA:82c7e631bb3cdc910f68e0081d67478d79c6982d) 2024-05-07T08:58:02.7188310Z Download action repository 'actions/cache@v4' (SHA:0c45773b623bea8c8e75f6c82b208c3cf94ea4f9) 2024-05-07T08:58:02.8972610Z Download action repository 'codecov/codecov-action@v4' (SHA:5ecb98a3c6b747ed38dc09f787459979aebb39be) 2024-05-07T08:58:03.6311820Z Complete job name: test (3.8, macos-latest) 2024-05-07T08:58:03.6813150Z ##[group]Run actions/checkout@v4 2024-05-07T08:58:03.6813780Z with: 2024-05-07T08:58:03.6813990Z repository: pypa/setuptools 2024-05-07T08:58:03.6814510Z token: *** 2024-05-07T08:58:03.6814820Z ssh-strict: true 2024-05-07T08:58:03.6815000Z ssh-user: git 2024-05-07T08:58:03.6815180Z persist-credentials: true 2024-05-07T08:58:03.6815400Z clean: true 2024-05-07T08:58:03.6815590Z sparse-checkout-cone-mode: true 2024-05-07T08:58:03.6815830Z fetch-depth: 1 2024-05-07T08:58:03.6816020Z fetch-tags: false 2024-05-07T08:58:03.6816210Z show-progress: true 2024-05-07T08:58:03.6816400Z lfs: false 2024-05-07T08:58:03.6816560Z submodules: false 2024-05-07T08:58:03.6816750Z set-safe-directory: true 2024-05-07T08:58:03.6816950Z env: 2024-05-07T08:58:03.6817100Z FORCE_COLOR: 1 2024-05-07T08:58:03.6817300Z PIP_DISABLE_PIP_VERSION_CHECK: true 2024-05-07T08:58:03.6817560Z PIP_NO_PYTHON_VERSION_WARNING: true 2024-05-07T08:58:03.6817850Z PIP_NO_WARN_SCRIPT_LOCATION: true 2024-05-07T08:58:03.6818150Z TOX_OVERRIDE: testenv.pass_env+=GITHUB_*,FORCE_COLOR 2024-05-07T08:58:03.6818460Z SETUPTOOLS_USE_DISTUTILS: local 2024-05-07T08:58:03.6818690Z ##[endgroup] 2024-05-07T08:58:03.9887890Z Syncing repository: pypa/setuptools 2024-05-07T08:58:03.9889020Z ##[group]Getting Git version info 2024-05-07T08:58:03.9889550Z Working directory is '/Users/runner/work/setuptools/setuptools' 2024-05-07T08:58:03.9890110Z [command]/opt/homebrew/bin/git version 2024-05-07T08:58:04.0649330Z git version 2.44.0 2024-05-07T08:58:04.0670900Z ##[endgroup] 2024-05-07T08:58:04.0681910Z Copying '/Users/runner/.gitconfig' to '/Users/runner/work/_temp/27009028-ce15-4b2c-a6ca-57d99f366c54/.gitconfig' 2024-05-07T08:58:04.0684330Z Temporarily overriding HOME='/Users/runner/work/_temp/27009028-ce15-4b2c-a6ca-57d99f366c54' before making global git config changes 2024-05-07T08:58:04.0685160Z Adding repository directory to the temporary git global config as a safe directory 2024-05-07T08:58:04.0686530Z [command]/opt/homebrew/bin/git config --global --add safe.directory /Users/runner/work/setuptools/setuptools 2024-05-07T08:58:04.0766090Z Deleting the contents of '/Users/runner/work/setuptools/setuptools' 2024-05-07T08:58:04.0768780Z ##[group]Initializing the repository 2024-05-07T08:58:04.0770910Z [command]/opt/homebrew/bin/git init /Users/runner/work/setuptools/setuptools 2024-05-07T08:58:04.0909040Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-07T08:58:04.0909980Z hint: is subject to change. To configure the initial branch name to use in all 2024-05-07T08:58:04.0910910Z hint: of your new repositories, which will suppress this warning, call: 2024-05-07T08:58:04.0928970Z hint: 2024-05-07T08:58:04.0929630Z hint: git config --global init.defaultBranch 2024-05-07T08:58:04.0929960Z hint: 2024-05-07T08:58:04.0930310Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-07T08:58:04.0930870Z hint: 'development'. The just-created branch can be renamed via this command: 2024-05-07T08:58:04.0931240Z hint: 2024-05-07T08:58:04.0931440Z hint: git branch -m 2024-05-07T08:58:04.0931830Z Initialized empty Git repository in /Users/runner/work/setuptools/setuptools/.git/ 2024-05-07T08:58:04.0932940Z [command]/opt/homebrew/bin/git remote add origin https://github.com/pypa/setuptools 2024-05-07T08:58:04.0970120Z ##[endgroup] 2024-05-07T08:58:04.0970580Z ##[group]Disabling automatic garbage collection 2024-05-07T08:58:04.0972640Z [command]/opt/homebrew/bin/git config --local gc.auto 0 2024-05-07T08:58:04.1015260Z ##[endgroup] 2024-05-07T08:58:04.1015650Z ##[group]Setting up auth 2024-05-07T08:58:04.1018940Z [command]/opt/homebrew/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-05-07T08:58:04.1058900Z [command]/opt/homebrew/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-05-07T08:58:04.1996770Z [command]/opt/homebrew/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-05-07T08:58:04.2064390Z [command]/opt/homebrew/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-05-07T08:58:04.3872980Z [command]/opt/homebrew/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-05-07T08:58:04.3999260Z ##[endgroup] 2024-05-07T08:58:04.3999800Z ##[group]Fetching the repository 2024-05-07T08:58:04.4004320Z [command]/opt/homebrew/bin/git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin +fa1e098721c4cd275bd1590b9ba0599a0f306fe9:refs/remotes/pull/4329/merge 2024-05-07T08:58:05.1801360Z From https://github.com/pypa/setuptools 2024-05-07T08:58:05.1802120Z * [new ref] fa1e098721c4cd275bd1590b9ba0599a0f306fe9 -> pull/4329/merge 2024-05-07T08:58:05.1850030Z ##[endgroup] 2024-05-07T08:58:05.1850420Z ##[group]Determining the checkout info 2024-05-07T08:58:05.1850820Z ##[endgroup] 2024-05-07T08:58:05.1853370Z [command]/opt/homebrew/bin/git sparse-checkout disable 2024-05-07T08:58:05.1902650Z [command]/opt/homebrew/bin/git config --local --unset-all extensions.worktreeConfig 2024-05-07T08:58:05.1941410Z ##[group]Checking out the ref 2024-05-07T08:58:05.1943470Z [command]/opt/homebrew/bin/git checkout --progress --force refs/remotes/pull/4329/merge 2024-05-07T08:58:05.3181550Z HEAD is now at fa1e098 Merge b1ef0c9c3f46eb458de87e0910fcd1c86663308b into dcfc833101a43dfa0621618fb04be0eee73e72c7 2024-05-07T08:58:05.3190790Z ##[endgroup] 2024-05-07T08:58:05.3245370Z [command]/opt/homebrew/bin/git log -1 --format='%H' 2024-05-07T08:58:05.3283820Z 'fa1e098721c4cd275bd1590b9ba0599a0f306fe9' 2024-05-07T08:58:05.3478190Z ##[group]Run actions/setup-python@v5 2024-05-07T08:58:05.3478490Z with: 2024-05-07T08:58:05.3478660Z python-version: 3.8 2024-05-07T08:58:05.3478870Z allow-prereleases: true 2024-05-07T08:58:05.3479090Z check-latest: false 2024-05-07T08:58:05.3479380Z token: *** 2024-05-07T08:58:05.3479560Z update-environment: true 2024-05-07T08:58:05.3479970Z env: 2024-05-07T08:58:05.3480110Z FORCE_COLOR: 1 2024-05-07T08:58:05.3480310Z PIP_DISABLE_PIP_VERSION_CHECK: true 2024-05-07T08:58:05.3480570Z PIP_NO_PYTHON_VERSION_WARNING: true 2024-05-07T08:58:05.3480820Z PIP_NO_WARN_SCRIPT_LOCATION: true 2024-05-07T08:58:05.3481110Z TOX_OVERRIDE: testenv.pass_env+=GITHUB_*,FORCE_COLOR 2024-05-07T08:58:05.3481400Z SETUPTOOLS_USE_DISTUTILS: local 2024-05-07T08:58:05.3481620Z ##[endgroup] 2024-05-07T08:58:05.4771080Z ##[group]Installed versions 2024-05-07T08:58:05.4809420Z Version ~3.8.0-0 was not found in the local cache 2024-05-07T08:58:06.2011650Z Version ~3.8.0-0 is available for downloading 2024-05-07T08:58:06.2012690Z Download from "https://github.com/actions/python-versions/releases/download/3.8.10-8879978422/python-3.8.10-darwin-arm64.tar.gz" 2024-05-07T08:58:07.2044120Z Extract downloaded archive 2024-05-07T08:58:07.2457640Z [command]/usr/bin/tar xz -C /Users/runner/work/_temp/69157ce9-92c4-4876-9516-111248db9fb9 -f /Users/runner/work/_temp/c3e23104-fdaa-42df-80e2-41ea21b3d7ce 2024-05-07T08:58:07.3063570Z Execute installation script 2024-05-07T08:58:07.3366940Z Check if Python hostedtoolcache folder exist... 2024-05-07T08:58:07.3468920Z Install Python binaries from prebuilt package 2024-05-07T08:58:32.8717770Z installer: Package name is Python 2024-05-07T08:58:32.8718440Z installer: Upgrading at base path / 2024-05-07T08:58:32.8718880Z installer: The upgrade was successful. 2024-05-07T08:58:32.8761300Z Create hostedtoolcach symlinks (Required for the backward compatibility) 2024-05-07T08:58:32.8764210Z Create Python 3.8.10 folder 2024-05-07T08:58:32.8887760Z Create additional symlinks (Required for the UsePythonVersion Azure Pipelines task and the setup-python GitHub Action) 2024-05-07T08:58:32.9016350Z Upgrading pip... 2024-05-07T08:58:33.1179460Z ##[error]Traceback (most recent call last): File "", line 6, in File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 203, in run_module mod_name, mod_spec, code = _get_module_details(mod_name) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 130, in _get_module_details spec = importlib.util.find_spec(mod_name) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/util.py", line 103, in find_spec return _find_spec(fullname, parent_path) File "", line 914, in _find_spec File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 79, in find_spec return method() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 100, in spec_for_pip if self.pip_imported_during_build(): File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 111, in pip_imported_during_build return any( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 112, in frame.f_globals['__file__'].endswith('setup.py') KeyError: '__file__' 2024-05-07T08:58:33.1221800Z ##[error]Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ensurepip/__main__.py", line 5, in sys.exit(ensurepip._main()) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ensurepip/__init__.py", line 206, in _main return _bootstrap( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ensurepip/__init__.py", line 125, in _bootstrap return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ensurepip/__init__.py", line 34, in _run_pip return subprocess.run([sys.executable, "-c", code], check=True).returncode File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmpzqwpe00u/setuptools-56.0.0-py3-none-any.whl\', \'/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmpzqwpe00u/pip-21.1.1-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmpzqwpe00u\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1. 2024-05-07T08:58:33.1249000Z ##[error]The process '/bin/bash' failed with exit code 1 2024-05-07T08:58:33.1492280Z Post job cleanup. 2024-05-07T08:58:33.3180890Z [command]/opt/homebrew/bin/git version 2024-05-07T08:58:33.3366480Z git version 2.44.0 2024-05-07T08:58:33.3424610Z Copying '/Users/runner/.gitconfig' to '/Users/runner/work/_temp/574425e4-74c3-4854-83ba-455f0f2d89e0/.gitconfig' 2024-05-07T08:58:33.3456380Z Temporarily overriding HOME='/Users/runner/work/_temp/574425e4-74c3-4854-83ba-455f0f2d89e0' before making global git config changes 2024-05-07T08:58:33.3464760Z Adding repository directory to the temporary git global config as a safe directory 2024-05-07T08:58:33.3468250Z [command]/opt/homebrew/bin/git config --global --add safe.directory /Users/runner/work/setuptools/setuptools 2024-05-07T08:58:33.3636490Z [command]/opt/homebrew/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-05-07T08:58:33.3686400Z [command]/opt/homebrew/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-05-07T08:58:33.5257740Z [command]/opt/homebrew/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-05-07T08:58:33.5311420Z http.https://github.com/.extraheader 2024-05-07T08:58:33.5333980Z [command]/opt/homebrew/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-05-07T08:58:33.5397880Z [command]/opt/homebrew/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-05-07T08:58:33.6496930Z Cleaning up orphan processes ```

Action version:

actions/setup-python@v5 - https://github.com/pypa/setuptools/actions/runs/8982558570/workflow?pr=4329#L73

Platform:

Runner type:

Tools version:

Python 3.8

Repro steps:
A description with steps to reproduce the issue. If your have a public example or repo to share, please provide the link.

https://github.com/pypa/setuptools/actions/runs/8982558570/job/24670435133?pr=4329#step:3:32

Expected behavior:

The setup-python action should successfully install Python 3.8 on macos-latest independently of the contents of the repository where the action is running.

Actual behavior:

The setup-python action fails to install Python 3.8 on macos-latest for the setuptools repository.


Please note that the issue is happening on the setuptools repository specifically, which may mean the following:


Also note that the suggestions in https://github.com/actions/setup-python/issues/850#issuecomment-2097496030 are not applicable. As far as I understand:

  1. Upgrading setuptools: The suggestion to install an updated version of setuptools before the step that is failing is not feasible. The step that is failing is the installation of Python itself. Therefore, it's not possible to update anything when Python is not installed yet[^1].

  2. Replacing setup.py with pyproject.toml: This suggestion also does not work and contains misleading and inaccurate information. The pyproject.toml is not more robust, it is just a different configuration format. Furthermore, setup.py, setup.cfg, and pyproject.toml are not mutually exclusive; they can all exist at the same time and serve different purposes. This is exactly the case in the repository experiencing the issue.

[^1]: Unless the users should always assume that the Python is pre-installed on the worker and how this affects the available actions. Please let me know if there is any documentation about this.

abravalheri commented 1 month ago

Update: even if I invert the order between checkout and setup-python so that setup-python is the very first action to run, it still fails: https://github.com/pypa/setuptools/pull/4343.

aparnajyothi-y commented 1 month ago

Hello @abravalheri, Thank you for creating this issue and we will look into it :)

ReenigneArcher commented 1 month ago

macos-latest was recently changed from macos-13 to macos-14. macos-13 was x86_64, while macos-14 is arm64.

This action fails on macos-latest from python 3.8, 3.9, and 3.10... but seems to be okay with python 3.11.

Run actions/setup-python@v5
  with:
    python-version: 3.10
    architecture: x64
    check-latest: false
    token: ***
    update-environment: true
    allow-prereleases: false
Installed versions
  Version 3.10 was not found in the local cache
  Version 3.10 is available for downloading
  Download from "https://github.com/actions/python-versions/releases/download/3.10.14-9004012336/python-3.10.14-darwin-x64.tar.gz"
  Extract downloaded archive
  /usr/bin/tar xz -C /Users/runner/work/_temp/d6bc2d46-6597-4924-9815-3d930552f60f -f /Users/runner/work/_temp/dae42830-86a3-4ec2-a5a4-02a135b36224
  Execute installation script
  Check if Python hostedtoolcache folder exist...
  Create Python 3.10.14 folder
  Copy Python binaries to hostedtoolcache folder
  Create additional symlinks (Required for the UsePythonVersion Azure Pipelines task and the setup-python GitHub Action)
  Upgrading pip...
  Error: dyld[8401]: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib
    Referenced from: <09857011-94D0-3FBA-9F9D-9FCE0E7366FF> /Users/runner/hostedtoolcache/Python/3.
  Error: 10.14/x64/bin/python3.10
    Reason: tried: '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)
  Error: ./setup.sh: line 53:  8401 Abort trap: 6           ./python -m ensurepip
  Error: The process '/bin/bash' failed with exit code 134

If you're relying on packages installed by homebrew or pre-installed to the runners, then the homebrew path is different on arm64 macs, verus the x86_64. The new path is /opt/homebrew instead of /usr/local.

abravalheri commented 1 month ago

Hi @ReenigneArcher, thank you very much for having a look on this.

Yeah, I need this specifically on Python 3.8 (I did notice it work fine on 3.11).

If you're relying on packages installed by homebrew or pre-installed to the runners, then the homebrew path is different on arm64 macs, verus the x86_64. The new path is /opt/homebrew instead of /usr/local.

I don't think I am. Even when actions/setup-python@v5 is the very first step to be executed in the job, with no custom code before, it fails. I never get past actions/setup-python@v5 to have the chance to use pre-installed packages or packages installed by homebrew...

You can see more details and the workflow file in https://github.com/pypa/setuptools/pull/4343.

ReenigneArcher commented 1 month ago

I don't think I am.

Sorry, I didn't mean "you"... I meant the code within the action.

I'm also experiencing the same issue as you, but wanted to share some information based other issues I've noticed with the new arm64 runners. I don't think github vetted the change to macos-14 (arm64) very well before making it the latest. Ideally all the actions they maintain should have been compatible with new architecture out of the box.