actions / setup-python

Set up your GitHub Actions workflow with a specific version of Python
MIT License
1.68k stars 538 forks source link

Fails on Rocky Linux distro, missing `lsb_release` #861

Closed garyo closed 2 months ago

garyo commented 4 months ago

Description: The following stanza fails on a CentOS 7 or Rocky Linux 8 Docker image in github, specifically aswf/ci-base:2023:

      - name: Set up python 3.11
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

Error log from build action:

Run actions/setup-python@v5
  with:
    python-version: 3.11
    cache: pip
    check-latest: false
    token: ***
    update-environment: true
    allow-prereleases: false
  env:
    BUILDTYPE_LC: release
    OSNAME: ubuntu
    GIT_COMMIT_ID: 641348a
    CONAN_PRESET: conan-release
    BUILD_DIR: build/Release
/usr/bin/docker exec  80999abf9153f55c7bed9a0[2](https://github.com/AcademySoftwareFoundation/openfx/actions/runs/8987538023/job/24686267871#step:6:2)f288fac8b4227[3](https://github.com/AcademySoftwareFoundation/openfx/actions/runs/8987538023/job/24686267871#step:6:3)9f71a56f2ecd135a272c725531 sh -c "cat /etc/*release | grep ^ID"
Installed versions
  Successfully set up CPython (3.11.[9](https://github.com/AcademySoftwareFoundation/openfx/actions/runs/8987538023/job/24686267871#step:6:9))
Error: Unable to locate executable file: lsb_release. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

Action version: v5

Platform:

Runner type:

Tools version: 3.11

Repro steps:
See above

Expected behavior: Not to fail; install Python

Actual behavior: Action fails, see above

aparnajyothi-y commented 4 months ago

Hello @garyo, Thank you for creating this issue and we will look into it :)

Nuru commented 4 months ago

I'm having the same problem with v5.1.0 on GitHub hosted ubuntu-22.04

      - uses: actions/setup-python@v5
        with:
          python-version: '3.9'
          cache: 'pip' # caching pip dependencies

Logs:

Run actions/setup-python@v5
  with:
    python-version: 3.9
    cache: pip
    check-latest: false
    token: ***
    update-environment: true
    allow-prereleases: false
/usr/bin/docker exec  e65a6dddc9afaa41c11c7df09f0617fc3082b82b82346802ca21d63e222dd840 sh -c "cat /etc/*release | grep ^ID"
Installed versions
  Successfully set up CPython (3.9.19)
Error: Unable to locate executable file: lsb_release. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
mahabaleshwars commented 4 months ago

Hello @garyo, would you mind trying to install lsb-release to see if it addresses the problem you're experiencing? Here are the commands for updating the apt package list and installing lsb-release:

- name : Apt-update 
  run: sudo apt-get update
- name: Install lsb-release
  run: sudo apt-get install -y lsb-release
garyo commented 4 months ago

It all works fine if that package is installed. It does not work on the image I mentioned (aswf/ci-base:2023) which does not install that by default. Images used for CI builds tend to be quite minimal to save space and speed up startup. Running apt-get update on every run will slow down CI builds. Perhaps if we know what you need from lsb_release there may be an alternative way to determine that on more minimal distros.

Nuru commented 4 months ago

This also happens on Alpine 3.17.3, Debian 11.5-slim, and Red Hat Enterprise Linux 8.9 (Ootpa).

Nuru commented 4 months ago

Apparently this is only a problem when using pip cache:

https://github.com/actions/setup-python/blob/82c7e631bb3cdc910f68e0081d67478d79c6982d/src/cache-distributions/pip-cache.ts#L69

https://github.com/actions/setup-python/blob/82c7e631bb3cdc910f68e0081d67478d79c6982d/src/utils.ts#L169-L172

mahabaleshwars commented 3 months ago

Hello @garyo, In this context, the lsb_release command retrieves the Linux operating system's name and version, facilitating the creation of unique cache keys for Python dependencies. This strategy ensures accurate caching and restoration of dependencies, considering the uniqueness of each combination of the operating system, Python version, and dependency file.

Can you consider using the actions/cache action to cache dependencies and speed up your workflows. In the case of Python, it's often used to cache pip packages. The key for the cache is often generated using the hash of requirements.txt (or other dependency files), and the Python version, not lsb_release. Here is an example:

- name: Cache pip
  uses: actions/cache@v4
  with:
    path: ${{ steps.pip-cache.outputs.dir }}
    key: <primaryKey>   # Replace with the computed primaryKey
    restore-keys: <restoreKey>   # Replace with the computed restoreKey

This will cache pip dependencies and reuse them in subsequent runs, reducing the time it takes to install packages. For any other inquiries or concerns pertaining to rocky-linux, please get in touch with the rocky-linux team.

garyo commented 3 months ago

Sure, I understand what lsb_release does. I expect it wouldn't be hard to get the same info in other ways when that is not available, but if you're not interested in that I'll look at other way to setup python on our CI containers.

pythoninthegrass commented 3 months ago

890 seems related. Will try using actions/cache@v4 in the interim.

mahabaleshwars commented 3 months ago

Hello @garyo, we are not planning to alter the usage of lsb_release for cache:pip. The provided solution utilizing actions/cache ensures pip dependencies are cached and reused in subsequent runs, thereby minimizing package installation time. Please feel free to reach us in case of any concerns.

mahabaleshwars commented 2 months ago

Hello @garyo, could you please confirm if the feedback addresses your concern?

garyo commented 2 months ago

I had to turn off caching because of this issue. Not great, but it works.

mahabaleshwars commented 2 months ago

Thank you @garyo for creating this issue. We are now closing it. Please feel free to reach out for any other issues.