Install PyYAML 6.0 from source results into failure: ERROR: No matching distribution found for wheel

[abhishek-das-gupta]

I am trying to instal PyYAML 6.0 using the source distribtuion instead of using the wheel. This is because my application can support multiple python versions(3.8, 3.9, 3.10) and I can't add corresponding wheels for each of them. Hence I'm building from source.

So This is the command within a make file which installs the pacakges from requirements.txt

TMPDIR=/root/cmf/build/sles15/agent-build/build.tmp /root/cmf/build/sles15/agent-build/env/bin/pip install -v --no-index -f file:///root/cmf/build/sles15/agent-build/ext-py --log /root/cmf/build/sles15/agent-build/pip.log --no-cache-dir -r requirements.txt

where the file:///root/cmf/build/sles15/agent-build/ext-py is the directory that contains the PyYAM 6.0 folder which I got from https://pypi.org/project/PyYAML/6.0/#files:~:text=Source%20Distribution-,PyYAML%2D6.0.tar.gz,-(125.0%20kB%20view

But when the above command runs within though a make command it fails. The /root/cmf/build/sles15/agent-build/pip.log shows the following:

2024-01-29T03:04:26,985 Processing /root/cmf/build/sles15/agent-build/ext-py/pyyaml-6.0.tar.gz
2024-01-29T03:04:27,152   Added pyyaml==6.0 from file:///root/cmf/build/sles15/agent-build/ext-py/pyyaml-6.0.tar.gz (from -r requirements.txt (line 22)) to build tracker '/tmp/pip-req-tracker-_g9eiqng'
2024-01-29T03:04:27,154   Created temporary directory: /tmp/pip-build-env-ry38jxa9
2024-01-29T03:04:27,156   Created temporary directory: /tmp/pip-standalone-pip-lmulhs74
2024-01-29T03:04:27,294   Running command /root/cmf/build/sles15/agent-build/env/bin/python /tmp/pip-standalone-pip-lmulhs74/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ry38jxa9/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: --no-index --find-links file:///root/cmf/build/sles15/agent-build/ext-py -- setuptools wheel Cython
2024-01-29T03:04:28,635   Using pip 21.2.4 from /tmp/pip-standalone-pip-lmulhs74/__env_pip__.zip/pip (python 3.10)
2024-01-29T03:04:28,699   Looking in links: file:///root/cmf/build/sles15/agent-build/ext-py
2024-01-29T03:04:28,725   Processing /root/cmf/build/sles15/agent-build/ext-py/setuptools-58.2.0-py3-none-any.whl
2024-01-29T03:04:28,736   ERROR: Could not find a version that satisfies the requirement wheel (from versions: none)
2024-01-29T03:04:28,736   ERROR: No matching distribution found for wheel
2024-01-29T03:04:28,787 WARNING: Discarding file:///root/cmf/build/sles15/agent-build/ext-py/pyyaml-6.0.tar.gz. Command errored out with exit status 1: /root/cmf/build/sles15/agent-build/env/bin/python /tmp/pip-standalone-pip-lmulhs74/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ry38jxa9/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: --no-index --find-links file:///root/cmf/build/sles15/agent-build/ext-py -- setuptools wheel Cython Check the logs for full command output.
2024-01-29T03:04:28,788 ERROR: Could not find a version that satisfies the requirement pyyaml==6.0 (from versions: 6.0)
2024-01-29T03:04:28,788 ERROR: No matching distribution found for pyyaml==6.0

Comparing the pip logs for some other pacakge,

2024-01-29T03:04:26,715 Processing /root/cmf/build/sles15/agent-build/ext-py/pyinotify-0.9.6.tar.gz
2024-01-29T03:04:26,729   Added pyinotify==0.9.6 from file:///root/cmf/build/sles15/agent-build/ext-py/pyinotify-0.9.6.tar.gz (from -r requirements.txt (line 20)) to build tracker '/tmp/pip-req-tracker-_g9eiqng'
2024-01-29T03:04:26,729     Running setup.py (path:/tmp/pip-install-ktw40g4c/pyinotify_cc70122b47e0457f85b41a6f495e13ed/setup.py) egg_info for package pyinotify
2024-01-29T03:04:26,729     Created temporary directory: /tmp/pip-pip-egg-info-y40aawhi
2024-01-29T03:04:26,729     Running command python setup.py egg_info
2024-01-29T03:04:26,932     running egg_info

Why the PyYam 6.0 not running the showing the log Running command python setup.py egg_info instead it is trying to install setuptools, wheels, CPython?

2024-01-29T03:04:27,294   Running command /root/cmf/build/sles15/agent-build/env/bin/python /tmp/pip-standalone-pip-lmulhs74/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ry38jxa9/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: --no-index --find-links file:///root/cmf/build/sles15/agent-build/ext-py -- setuptools wheel Cython

Here is the pip which is getting reffered.

py310:~/cmf # /root/cmf/build/sles15/agent-build/env/bin/pip list
Package           Version
----------------- -------
packaging         21.0
pip               21.2.4
pyparsing         2.4.7
python-libtorrent 1.1.5
setuptools        58.2.0
setuptools-scm    6.3.2
tomli             1.2.1
wheel             0.42.0
WARNING: You are using pip version 21.2.4; however, version 23.3.2 is available.
You should consider upgrading via the '/root/cmf/build/sles15/agent-build/env/bin/python -m pip install --upgrade pip' command.
py310:~/cmf # /root/cmf/build/sles15/agent-build/env/bin/pip --version
pip 21.2.4 from /root/cmf/build/sles15/agent-build/env/lib64/python3.10/site-packages/pip (python 3.10)

Please suggest how do I debug this and get the source distribution to install instead. Thanks!

[nitzmahone]

It sounds like you might have a number of misunderstandings about how Python packaging, build, and dependency installation work. Generally, each Python version has its own version-specific site-packages dir, so as long as you invoke each Python version's pip correctly (eg, use python3.10 -m pip instead of the pip script wrapper to ensure you're running the right pip to manage your python3.10 env, python3.11 -m pip for 3.11, etc), you can just install the right pre-built wheel to each version's environment. Building from source isn't going to help you- modern versions of pip (under the covers) always try to build a wheel to install from source anyway, so you're going to end up with roughly the same thing (sans all the testing that our distributed binary wheels get).

If you're trying to do something custom like mashing all the different Python versions' deps into a single directory and use sys.path hackery to make it load, well, good luck with that. It might actually work for PyYAML today if you manually extract the wheels and just put all the version-specific binaries next to each other (Python will ignore the foreign versions), but that's not "supported" and could break at any time.

In any case, this isn't a deficiency of PyYAML, so I'm closing this issue.