Open SuperValou opened 11 months ago
This is expected, future
doesn't have any wheels on PyPI. See https://pypi.org/project/future/#file.
I'm not really familiar with the implications of having a tar.gz
instead of a .whl
file. Isn't pip able to handle it anyway? I guess not (obviously) but as pip install future
is possible it feels weird to have to "know" if a package is rez-pipable or not. I'll investigate more on this topic to better understand the issue.
The thing is, any package having a dependency to this sort of package becomes un-rez-pipable as well. As an example, rez-piping lucidity
fails due to its dependency to future
:
rez-pip2 lucidity
INFO Installing requested packages for Python 3.7.7
ERROR: Could not find a version that satisfies the requirement future (from lucidity) (from versions: none)
ERROR: No matching distribution found for future
rez_pip.exceptions.PipError: Failed to run pip command: '.\\python.exe c:\\rez-pip\\lib\\site-packages\\rez_pip\\data\\pip.pyz install -q lucidity --disable-pip-version-check --dry-run --ignore-installed --python-version=3.7.7 --only-binary=:all: --target=/tmp/asd --disable-pip-version-check --report C:\\Users\\V240D~1.ESM\\AppData\\Local\\Temp\\pip-install-outputsnfkn57v'
Pip reported this:
ERROR: Could not find a version that satisfies the requirement future (from lucidity) (from versions: none)
ERROR: No matching distribution found for future
Is it an expected behaviour of rez-pip?
We use pip to get the full list of dependencies. That's the only thing we use from pip. We don't install packages with pip.
The tar file is called the source distribution (sdist). The source distribution doesn't contain any metadata that can be used to get the list of dependencies. So when you pip install future
, pip will actually download the sdist, build a wheel from it, then get the dependencies from that freshly built wheel.
--only-binary=:all:
tells pip to not do that and ignore sdists entirely.
Thanks a lot for the clarification, now that I dug a bit into this issue on my side as well I better understand this topic.
From what I understand, we use the download url of what is returned by pip (around here), then the whl gets downloaded (here) and unzipped/installed (here). The installation part assumes we're installing a whl file and that's why sdist aren't currently supported if I understand corretly (at least that's where everything starts to break if I try to remove --only-binary=:all:
and --python-version
from the command and try to run rez-pip future
).
Couldn't we think about extending the ability of rez-pip so it can install sdists as well? Maybe I could rename the issue to "Unable to install sdist packages", what do you think?
Couldn't we think about extending the ability of rez-pip so it can install sdists as well?
I unfortunately don't think we can with the current architecture. At least for cases like rez-pip2 lucidity
where one of the transitive dependencies doesn't have a wheel.
There is a very specific reason why sdists aren't supported and it's currently partially (or implicitly) documented at https://rez-pip.readthedocs.io/en/latest/faq.html#why-can-t-it-install-python-2-packages.
As you can see, rez-pip does not rely on pip to install the packages. We only use pip to resolve which package we need to download and then “manually” install the wheels. This allows to fix the notorious shebang problem with pip. Pip always bakes the full Python interpreter path into the console scripts shebang. The problem is that baking the full path goes against the idea of rez. In rez, we want the use the resolve python package to executable console scripts, not the Python interpreter that was used to install the package.
We could maybe support passing an sdist to rez-pip2 (rez-pip2 /path/to/dsist.tar.gz
), but this gets into the build territory, which I tried to avoid like mentioned in https://rez-pip.readthedocs.io/en/latest/transition.html#local-and-editable-installs.
For pure python packages, this seems overkill to me, considering how easy it is to create wheels (you can either use pip wheel <package> --no-deps
or https://github.com/pypa/build).
I'll see if I can think about something that could simplify everyone's life, but I don't know when or if I'll be able to come up with a clear answer/plan for this.
Thanks for the detailed clarifications, I now see your point and agree that rez-pip should probably not go down the build route as well.
I'll attempt to build a future
whl and see what kind of workflow I can come up with to be able to rez-pip lucidity
somehow. Maybe doing something as naive as rez-pip2 /path/to/future/wheel lucidity
actually works, I'll keep you posted.
When using rez-pip to install the future package, the install fails.
Environment
To Reproduce
rez env python-3.7.7 -- python -m venv C:\rez-pip
to generate a virtualenv that will host the rez-pip packagecd C:\rez-pip\Scripts
then.\activate
to enter the virtualenvpython -m pip install rez-pip
to install rez-pip in the virtualenvcd C:\rez-pip\Scripts
then.\activate
to enter the virtualenv ofrez-pip
.\rez-pip2.exe future
Pip reported this:
ERROR: Could not find a version that satisfies the requirement future (from versions: none) ERROR: No matching distribution found for future
(rez-pip) PS C:\rez-pip\Scripts> .\python.exe c:\rez-pip\lib\site-packages\rez_pip\data\pip.pyz install future --disable-pip-version-check --dry-run --ignore-installed --python-version=3.7.7 --only-binary=:all: --target=/tmp/
ERROR: Could not find a version that satisfies the requirement future (from versions: none) ERROR: No matching distribution found for future
(rez-pip) PS C:\rez-pip\Scripts> .\python.exe c:\rez-pip\lib\site-packages\rez_pip\data\pip.pyz install future --disable-pip-version-check --dry-run --ignore-installed --target=/tmp/
Collecting future Using cached future-0.18.3-py3-none-any.whl Would install future-0.18.3