No module named pip for x86 manylinux_2_28

[jennydaman]

My actions contains a section like:

  maturin-linux:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target: [x86_64, aarch64, armv7]
    steps:
      - uses: actions/checkout@v4
      - name: Build wheels
        uses: PyO3/maturin-action@v1
        with:
          target: ${{ matrix.target }}
          args: --release --strip --locked --out dist --manifest-path chrs/Cargo.toml
          # sscache seems to be broken
          # sccache: 'true'
          manylinux: manylinux_2_28
      - name: Upload wheels
        uses: actions/upload-artifact@v3
        with:
          name: wheels
          path: dist

If I set sccache: 'true', I get this error:

/usr/bin/python3: No module named pip
  Error: The process '/usr/bin/docker' failed with exit code 1
      at ExecState._setResult (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1702:25)
      at ExecState.CheckComplete (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1685:18)
      at ChildProcess.<anonymous> (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1579:27)
      at ChildProcess.emit (node:events:514:28)
      at maybeClose (node:internal/child_process:1105:16)
      at ChildProcess._handle.onexit (node:internal/child_process:305:5)

https://github.com/FNNDSC/chrs/actions/runs/8243560850/job/22544352067

With sccache commented out, the builds are successful. https://github.com/FNNDSC/chrs/actions/runs/8243573628/job/22544388834

[NfNitLoop]

I'm getting this error too. But, I think the error in sscache is just a duplicate of an error that happens earlier in the build:

Run PyO3/maturin-action@v1
  with:
    target: x86_64
    args: --release --out dist
    working-directory: components/kernel/nxd_py
    sccache: true
    manylinux: [2](https://github.com/nextdata-tech/nxd/actions/runs/8668310817/job/23773832284#step:4:2)_28
    token: ***
    command: build
  env:
    NXD_PY_DIR: components/kernel/nxd_py
    pythonLocation: /opt/hostedtoolcache/Python/[3](https://github.com/nextdata-tech/nxd/actions/runs/8668310817/job/23773832284#step:4:3).10.14/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.10.1[4](https://github.com/nextdata-tech/nxd/actions/runs/8668310817/job/23773832284#step:4:4)/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.14/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.14/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.14/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.14/x64/lib
Found maturin version requirement maturin>=1.4,<2.0 specified in pyproject.toml
Found maturin release from manifest: v1.[5](https://github.com/nextdata-tech/nxd/actions/runs/8668310817/job/23773832284#step:4:5).1

Pull Docker image
  Using quay.io/pypa/manylinux_2_28_x86_64:latest Docker image
  /usr/bin/docker pull quay.io/pypa/manylinux_2_28_x86_64:latest
  latest: Pulling from pypa/manylinux_2_28_x86_64
  [completes successfully]

Install maturin
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed

    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  100  9.7M  100  9.7M    0     0  27.8M      0 --:--:-- --:--:-- --:--:-- 27.8M
  maturin 1.5.1
  /usr/bin/python3: No module named pip

So, whatever image is being pulled (quay.io/pypa/manylinux_2_28_x86_64:latest) doesn't seem to include pip?

The other branch of my matrix build works fine with ghcr.io/rust-cross/manylinux_2_28-cross:aarch64.

Install maturin
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed

    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  100  9.7M  100  9.7M    0     0  17.3M      0 --:--:-- --:--:-- --:--:-- 17.3M
  maturin 1.5.1
  Requirement already satisfied: cffi in /usr/local/lib/python3.10/dist-packages (1.16.0)
  Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi) (2.21)
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

  Notice:  A new release of pip is available: 23.3.1 -> 24.0
  Notice:  To update, run: python3 -m pip install --upgrade pip

[NfNitLoop]

I'd recommend we rename this bug to "No module named pip for x86 manylinux_2_28".

[NfNitLoop]

Interesting. trying out the broken image:

> docker run -ti quay.io/pypa/manylinux_2_28_x86_64:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
[root@9052a380cca1 /]# python3 -m pip
/usr/bin/python3: No module named pip
[root@9052a380cca1 /]# ls -l $(which python3)
lrwxrwxrwx 1 root root 25 Mar 18 06:19 /usr/bin/python3 -> /etc/alternatives/python3
[root@9052a380cca1 /]# ls -l /etc/alternatives/python3
lrwxrwxrwx 1 root root 19 Mar 18 06:19 /etc/alternatives/python3 -> /usr/bin/python3.11
[root@9052a380cca1 /]# python3.11 -m pip

Usage:
  /usr/local/bin/python3.11 -m pip <command> [options]

Commands:
  install                     Install packages.
  [etc]

[messense]

Not sure why they changed that. Does it have ensurepip? Or maybe we need to yum install python3-pip.

[orf]

To help with working around this, I've made this: https://github.com/PyO3/maturin-action/pull/264

[marcpabst]

Installing python3-pip did not work and somehow installed an outdated version of Python 3 and pip for me. python3 -m ensurepip did however work.