Closed gGonz closed 3 years ago
Probably related:
With 20.3.2, pip install setuptools --upgrade
is iterating all versions and is taking long minutes to not do anything.
With 20.3.1 it finishes in a few seconds.
@pradyunsg, @uranusjr, and @NoahGorny this is a really bad bug 😬
I just ran into this myself when trying to upgrade my version of pandas (1.1.3) to the latest version (currently 1.1.5). And then saw that it did the same thing when trying to upgrade pip itself.
Out of morbid curiosity, I let the pandas upgrade run its course inside a fresh virtual environment. It took over 35 minutes and downloaded over 650 MB of old packages into the cache directory: https://pastebin.com/V6kg6tTX
Edit: Moved code to as per @omry.
It should be noted, it's not an infinite loop...
Here's the full log of pip 20.3.2 trying to upgrade itself (adds 157 MB of old versions into the cache): https://pastebin.com/AFny1enP
Edit: Moved code to as per @omry.
can you use pastebin when pasting very long pastes? it makes looking for comments here very painful.
Oh my god, Please fix it ASAP. Thank you.
Gosh...Plz...CI System bombed by pip...
This is the mitigation I'm using until the bug is fixed.
python -m pip install 'pip==20.3.1'
@wyqsmith
Exactly
I did some quick research, and this bug seems to have been introduced by the packaging 20.8 vendor upgrade. This was unfortunately not caught due to the PyPI search API was causing failures on CI, and we missed the test failure in a sea of red. (See update below)
Unfortunately I cannot do anything right now since I don’t have access to PyPI versions, but will try to contact other members who do when they are available. (20.3.2 has now been yanked from PyPI. Thanks @xavfernandez!)
I did some bisecting, and the issue is indeed from a resolver change, specifically in d45541c8f3b3d87ae55a08d7021e8e879293285c.
The issue was not identified in tests because it requires a set of conditions to trigger (I have not figured out the exact conditions).
This is the mitigation I'm using until the bug is fixed.
python -m pip install 'pip==20.3.1'
ValueError: Unable to find resource t64.exe in package pip._vendor.distlib
@tiantian1645 This is a common issue when you try to roll back a broken pip version. See https://github.com/pypa/pip/issues/8450#issuecomment-671451859 for steps to rescure.
People affected by this: Could you try
pip install --force-reinstall "pip @ https://github.com/uranusjr/pip/archive/new-resolver-do-not-eagerly-consume-in-upgrade-mode.zip"
and see if it behaves correctly?
seem to be good... but with version 21.0.dev0? I can not check if this fix "no ignore yanked" bug... can you make this package with 20.3.1 number of version to check that?
You can temporary workaround it by using the old resolver:
pip install --upgrade pip --use-deprecated=legacy-resolver
pip install -U pip --use-deprecated=legacy-resolver
work flawesly...
Since this is not an infinite loop, please change the title of this issue to prevent further duplicates.
Oh boi. This is "fun". Sorry for the disruption folks!
Let's revert https://github.com/pypa/pip/commit/d45541c8f3b3d87ae55a08d7021e8e879293285c, and then take things from there?
Regarding workarounds: There is also the option of putting this in the pip.ini
/pip.conf
in case you can not change the pip command:
[install]
use-deprecated=legacy-resolver
Be aware that devpi does not support PEP 592 in latest stable version of server and yanked version will be installed if pypi mirroring is enabled.
I have a warning that if you upgrade pip twice in the same venv then you will still end with this bad version:
$ python3 -m venv /tmp/venv
$ /tmp/venv/bin/pip install pip --upgrade
Collecting pip
Using cached https://files.pythonhosted.org/packages/ab/11/2dc62c5263d9eb322f2f028f7b56cd9d096bb8988fcf82d65fa2e4057afe/pip-20.3.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 19.3.1
Uninstalling pip-19.3.1:
Successfully uninstalled pip-19.3.1
Successfully installed pip-20.3.1
$ /tmp/venv/bin/pip install pip --upgrade
Requirement already satisfied: pip in /tmp/venv/lib/python3.8/site-packages (20.3.1)
Collecting pip
Using cached pip-20.3.2-py2.py3-none-any.whl (1.5 MB)
WARNING: The candidate selected for download or install is a yanked version: 'pip' candidate (version 20.3.2 at https://files.pythonhosted.org/packages/3d/0c/01014c0442830eb38d6baef0932fdcb389279ce74295350ecb9fe09e048a/pip-20.3.2-py2.py3-none-any.whl#sha256=8d779b6a85770bc5f624b5c8d4d922ea2e3cd9ce6ee92aa260f12a9f072477bc (from https://pypi.org/simple/pip/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*))
Reason for being yanked: <none given>
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.3.1
Uninstalling pip-20.3.1:
Successfully uninstalled pip-20.3.1
Successfully installed pip-20.3.2
It would be safer if 20.3.3 was released with just the bad change reverted instead of using this "yanked" feature. Some mirrors seem to ignore this "yanked" feature too - I've noticed that Artifactory does.
It would be safer if 20.3.3 was released with just the bad change reverted instead of using this "yanked" feature. Some mirrors seem to ignore this "yanked" feature too - I've noticed that Artifactory does.
Yup. I'll cut a 20.3.3 after https://github.com/pypa/pip/pull/9293 is merged. :)
Could the PyPI Service Degradation incident be related?
The degradation affects the XML-RPC endpoints, which pip install
does not use. pip currently only uses it in pip search
(and is already planning to remove it).
With #9293 merged, I think this is fixed in master at this time.
I'll cut a release in a ~30 minutes or so.
Confirming that with the new pip version pip==20.3.3
, the problem is fixed. (tested on windows)
for clarity, if you are currently affected by this, and your pip is barfing endlessly, run whichever corresponds to your permissions and pip version:
pip install 'pip==20.3.3'
# or
sudo pip install 'pip==20.3.3'
# or
pip3 install 'pip==20.3.3'
# or
sudo pip3 install 'pip==20.3.3'
pip 20.3.3 is still dumping 23 Gb of dependencies:
INFO: pip is looking at multiple versions of sympy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of statsmodels to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of redis to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pyjwt to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pika to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of parsimonious to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pandas to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of marshmallow to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of language-tags to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of html5lib to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask-testing to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask-swagger-ui to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask-cors to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask-compress to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask-apispec to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cython to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of apispec to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of modelling2 to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of langdetect to determine which version is compatible with other requirements. This could take a while.
Example:
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
Using cached conllu-4.2-py2.py3-none-any.whl (14 kB)
Using cached conllu-4.1-py2.py3-none-any.whl (14 kB)
Using cached conllu-4.0-py2.py3-none-any.whl (13 kB)
Using cached conllu-3.1.1-py2.py3-none-any.whl (14 kB)
Using cached conllu-3.1-py2.py3-none-any.whl (14 kB)
Using cached conllu-3.0-py2.py3-none-any.whl (14 kB)
INFO: pip is looking at multiple versions of conllu to determine which version is compatible with other requirements. This could take a while.
Using cached conllu-2.3.2-py2.py3-none-any.whl (13 kB)
Using cached conllu-2.3-py2.py3-none-any.whl (13 kB)
Using cached conllu-2.2.2-py2.py3-none-any.whl (13 kB)
Using cached conllu-2.2.1-py2.py3-none-any.whl (12 kB)
Using cached conllu-2.2-py2.py3-none-any.whl (12 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
Using cached conllu-2.1.1-py2.py3-none-any.whl (12 kB)
Using cached conllu-2.1-py2.py3-none-any.whl (12 kB)
Using cached conllu-2.0-py2.py3-none-any.whl (10 kB)
Using cached conllu-1.5-py2.py3-none-any.whl (9.8 kB)
Using cached conllu-1.4.1-py2.py3-none-any.whl (9.5 kB)
Using cached conllu-1.4-py2.py3-none-any.whl (9.5 kB)
Using cached conllu-1.3.2-py2.py3-none-any.whl (9.3 kB)
Using cached conllu-1.3.1-py2.py3-none-any.whl (9.3 kB)
Using cached conllu-1.3-py2.py3-none-any.whl (9.3 kB)
Using cached conllu-1.2.3-py2.py3-none-any.whl (9.2 kB)
Using cached conllu-1.2.2-py2.py3-none-any.whl (9.2 kB)
Using cached conllu-1.2.1-py2.py3-none-any.whl (9.1 kB)
Using cached conllu-1.2-py2.py3-none-any.whl (9.1 kB)
Using cached conllu-1.1-py2.py3-none-any.whl (8.8 kB)
Using cached conllu-1.0.1-py2.py3-none-any.whl (8.3 kB)
Using cached conllu-1.0-py2.py3-none-any.whl (8.2 kB)
Using cached conllu-0.11-py2.py3-none-any.whl (6.8 kB)
Using cached conllu-0.10.7-py2.py3-none-any.whl (6.3 kB)
Using cached conllu-0.10.6-py2.py3-none-any.whl (6.2 kB)
Using cached conllu-0.10.5-py2.py3-none-any.whl (6.2 kB)
Using cached conllu-0.10.4-py2.py3-none-any.whl (6.2 kB)
Using cached conllu-0.10.3-py3-none-any.whl (5.4 kB)
Using cached conllu-0.10.2-py3-none-any.whl (5.4 kB)
Using cached conllu-0.10.1.tar.gz (5.4 kB)
Using cached conllu-0.10.tar.gz (5.3 kB)
Using cached conllu-0.9.tar.gz (4.5 kB)
Using cached conllu-0.8.tar.gz (4.5 kB)
Using cached conllu-0.7.tar.gz (4.4 kB)
Using cached conllu-0.6.1.tar.gz (4.3 kB)
Using cached conllu-0.6.tar.gz (4.2 kB)
Using cached conllu-0.5.tar.gz (3.8 kB)
Using cached conllu-0.4.tar.gz (2.1 kB)
Using cached conllu-0.3.tar.gz (2.0 kB)
Using cached conllu-0.2-py2.py3-none-any.whl (8.2 kB)
Using cached conllu-0.1.tar.gz (1.6 kB)
This is not scalable. Before things worked fine without downloading so many packages.
pip 20.3.3 works fine for me on macOS 11.1. @pabloa I ran pip cache purge
after pip install 'pip==20.3.3'
. Just running pip install -U pip
lead to pip trying to install its whole ancestry since it still was 20.3.2
Yes, the issue is triggered by the -U
(or --upgrade
) flag. So the escape plan is to install another pip version without that flag.
I’ll close this since the eager download issue seems fixed. The “pip downloads many versions” issue is tracked in #9187.
I tested with an existent and a new virtual environment. The last report is from a CI server that deletes everything and creates a new virtual environment.
Yeah, seems to be fixed. Going from 20.3.1 to 20.3.3 with pip install -U pip
and doing the same over 20.3.3 works as expected. Thanks!
Thanks for the fix on this... I was starting to think I was crazy.
And for the search engines: pip downloading lots of old versions
Does anyone know how to install 20.3.3 if you have 20.3.2? I try to upgrade and it keeps pulling every version.
Nevermind, pip install 'pip==20.3.3'
(https://github.com/pypa/pip/issues/9284#issuecomment-745444763) worked perfectly.
FYI (and to capture the experience), I also got the slightly deranged experience when attempting to upgrade from 20.3.2:
Good suggestion to use pip==20.3.3
when upgrading.
this still happens on the latest version, thats crazy
(venv) oli@lucy:~/Downloads/virtual_webcam_background-master$ pip install -r requirements.txt
Collecting tf-nightly==2.5.0.dev20210316
Using cached tf_nightly-2.5.0.dev20210316-cp39-cp39-manylinux2010_x86_64.whl (427.5 MB)
Collecting numpy==1.19.5
Using cached numpy-1.19.5-cp39-cp39-manylinux2010_x86_64.whl (14.9 MB)
Collecting scipy==1.6.1
Using cached scipy-1.6.1-cp39-cp39-manylinux1_x86_64.whl (27.3 MB)
Collecting pillow==8.1.2
Using cached Pillow-8.1.2-cp39-cp39-manylinux1_x86_64.whl (2.2 MB)
Collecting pyyaml==5.4.1
Using cached PyYAML-5.4.1-cp39-cp39-manylinux1_x86_64.whl (630 kB)
Collecting pyfakewebcam==0.1.0
Using cached pyfakewebcam-0.1.0.tar.gz (12 kB)
Collecting opencv-python==4.5.1.48
Using cached opencv_python-4.5.1.48-cp39-cp39-manylinux2014_x86_64.whl (50.4 MB)
Collecting tfjs-graph-converter==1.4.2
Using cached tfjs_graph_converter-1.4.2-py3-none-any.whl (32 kB)
Collecting flatbuffers~=1.12.0
Using cached flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting termcolor~=1.1.0
Using cached termcolor-1.1.0.tar.gz (3.9 kB)
Collecting typing-extensions~=3.7.4
Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting opt-einsum~=3.3.0
Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Requirement already satisfied: six~=1.15.0 in /usr/lib/python3/dist-packages (from tf-nightly==2.5.0.dev20210316->-r requirements.txt (line 1)) (1.15.0)
Collecting astunparse~=1.6.3
Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting tb-nightly~=2.5.0.a
Using cached tb_nightly-2.5.0a20210316-py3-none-any.whl (6.0 MB)
Collecting wrapt~=1.12.1
Using cached wrapt-1.12.1.tar.gz (27 kB)
Collecting gast==0.4.0
Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting google-pasta~=0.2
Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting wheel~=0.35
Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Collecting keras-preprocessing~=1.1.2
Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting h5py~=3.1.0
Using cached h5py-3.1.0-cp39-cp39-manylinux1_x86_64.whl (4.4 MB)
Collecting grpcio~=1.34.0
Using cached grpcio-1.34.1-cp39-cp39-manylinux2014_x86_64.whl (4.0 MB)
Collecting protobuf>=3.9.2
Using cached protobuf-3.15.6-cp39-cp39-manylinux1_x86_64.whl (1.0 MB)
Collecting absl-py~=0.10
Using cached absl_py-0.12.0-py3-none-any.whl (129 kB)
Collecting keras-nightly~=2.5.0.dev
Using cached keras_nightly-2.5.0.dev2021031600-py2.py3-none-any.whl (1.3 MB)
Collecting tf-estimator-nightly~=2.5.0.dev
Using cached tf_estimator_nightly-2.5.0.dev2021031601-py2.py3-none-any.whl (462 kB)
Collecting tensorflowjs>=1.5.2
Using cached tensorflowjs-3.3.0-py3-none-any.whl (63 kB)
Collecting werkzeug>=0.11.15
Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
Using cached google_auth_oauthlib-0.4.3-py2.py3-none-any.whl (18 kB)
Collecting google-auth<2,>=1.6.3
Using cached google_auth-1.27.1-py2.py3-none-any.whl (136 kB)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/lib/python3/dist-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0.dev20210316->-r requirements.txt (line 1)) (2.25.1)
Requirement already satisfied: setuptools>=41.0.0 in ./venv/lib/python3.9/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0.dev20210316->-r requirements.txt (line 1)) (44.1.1)
Collecting tensorboard-plugin-wit>=1.6.0
Using cached tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
Collecting markdown>=2.6.8
Using cached Markdown-3.3.4-py3-none-any.whl (97 kB)
Collecting tensorboard-data-server<0.5.0,>=0.4.0
Using cached tensorboard_data_server-0.4.0-py3-none-manylinux2010_x86_64.whl (3.9 MB)
Collecting cachetools<5.0,>=2.0.0
Using cached cachetools-4.2.1-py3-none-any.whl (12 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting requests-oauthlib>=0.7.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
Collecting tensorflow-hub<0.10,>=0.7.0
Using cached tensorflow_hub-0.9.0-py2.py3-none-any.whl (103 kB)
Collecting tensorflowjs>=1.5.2
Using cached tensorflowjs-3.2.0-py3-none-any.whl (63 kB)
Using cached tensorflowjs-3.1.0-py3-none-any.whl (63 kB)
Using cached tensorflowjs-3.0.0-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.8.5-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.8.4-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.8.3-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.8.2-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.8.1-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.8.0-py3-none-any.whl (63 kB)
Using cached tensorflowjs-2.7.0-py3-none-any.whl (62 kB)
Using cached tensorflowjs-2.6.0-py3-none-any.whl (61 kB)
Using cached tensorflowjs-2.5.0-py3-none-any.whl (61 kB)
Using cached tensorflowjs-2.4.0-py3-none-any.whl (61 kB)
Collecting tensorflow-hub==0.7.0
Using cached tensorflow_hub-0.7.0-py2.py3-none-any.whl (89 kB)
Collecting tensorflowjs>=1.5.2
Using cached tensorflowjs-2.3.0-py3-none-any.whl (61 kB)
Using cached tensorflowjs-2.1.0-py3-none-any.whl (60 kB)
Using cached tensorflowjs-2.0.1.post1-py3-none-any.whl (60 kB)
Using cached tensorflowjs-2.0.1-py3-none-any.whl (60 kB)
Using cached tensorflowjs-2.0.0-py3-none-any.whl (57 kB)
Using cached tensorflowjs-1.7.4.post1-py3-none-any.whl (57 kB)
Using cached tensorflowjs-1.7.4-py3-none-any.whl (57 kB)
Using cached tensorflowjs-1.7.3-py3-none-any.whl (57 kB)
Using cached tensorflowjs-1.7.2-py3-none-any.whl (57 kB)
Using cached tensorflowjs-1.6.0-py3-none-any.whl (57 kB)
Using cached tensorflowjs-1.5.2-py3-none-any.whl (56 kB)
INFO: pip is looking at multiple versions of tensorboard-plugin-wit to determine which version is compatible with other requirements. This could take a while.
Collecting tensorboard-plugin-wit>=1.6.0
Using cached tensorboard_plugin_wit-1.7.0-py3-none-any.whl (779 kB)
Using cached tensorboard_plugin_wit-1.6.0.post3-py3-none-any.whl (777 kB)
Using cached tensorboard_plugin_wit-1.6.0.post2-py3-none-any.whl (775 kB)
INFO: pip is looking at multiple versions of tensorboard-data-server to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of rsa to determine which version is compatible with other requirements. This could take a while.
Collecting rsa<5,>=3.1.4
Using cached rsa-4.7.1-py3-none-any.whl (36 kB)
Using cached rsa-4.7-py3-none-any.whl (34 kB)
Using cached rsa-4.6-py3-none-any.whl (47 kB)
Using cached rsa-4.5-py2.py3-none-any.whl (36 kB)
Using cached rsa-4.4.1-py2.py3-none-any.whl (33 kB)
Using cached rsa-4.3-py2.py3-none-any.whl (36 kB)
Using cached rsa-4.2.tar.gz (46 kB)
Using cached rsa-4.1-py3-none-any.whl (32 kB)
Using cached rsa-4.0-py2.py3-none-any.whl (38 kB)
Using cached rsa-3.4.2-py2.py3-none-any.whl (46 kB)
Using cached rsa-3.4.1-py2.py3-none-any.whl (46 kB)
Using cached rsa-3.4-py2.py3-none-any.whl (46 kB)
Using cached rsa-3.3-py2.py3-none-any.whl (44 kB)
Using cached rsa-3.2.3-py2.py3-none-any.whl (44 kB)
Using cached rsa-3.2.2-py2.py3-none-any.whl (44 kB)
Using cached rsa-3.2-py2.py3-none-any.whl (43 kB)
Using cached rsa-3.1.4.tar.gz (36 kB)
INFO: pip is looking at multiple versions of tensorboard-plugin-wit to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of tensorboard-data-server to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of rsa to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of oauthlib to determine which version is compatible with other requirements. This could take a while.
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.0.2-py2.py3-none-any.whl (143 kB)
Using cached oauthlib-3.0.1-py2.py3-none-any.whl (142 kB)
Using cached oauthlib-3.0.0-py2.py3-none-any.whl (142 kB)
INFO: pip is looking at multiple versions of requests-oauthlib to determine which version is compatible with other requirements. This could take a while.
Collecting requests-oauthlib>=0.7.0
Using cached requests_oauthlib-1.2.0-py2.py3-none-any.whl (22 kB)
Using cached requests_oauthlib-1.1.0-py2.py3-none-any.whl (21 kB)
Using cached requests_oauthlib-1.0.0-py2.py3-none-any.whl (21 kB)
Using cached requests_oauthlib-0.8.0-py2.py3-none-any.whl (23 kB)
Using cached requests_oauthlib-0.7.0-py2.py3-none-any.whl (22 kB)
INFO: pip is looking at multiple versions of oauthlib to determine which version is compatible with other requirements. This could take a while.
Collecting oauthlib<3.0.0,>=2.1.0
Using cached oauthlib-2.1.0-py2.py3-none-any.whl (121 kB)
Collecting oauthlib>=0.6.2
Using cached oauthlib-2.0.7-py2.py3-none-any.whl (124 kB)
Using cached oauthlib-2.0.6.tar.gz (127 kB)
Using cached oauthlib-2.0.5.tar.gz (129 kB)
Using cached oauthlib-2.0.4.tar.gz (127 kB)
Using cached oauthlib-2.0.3.tar.gz (127 kB)
Using cached oauthlib-2.0.2.tar.gz (125 kB)
Using cached oauthlib-2.0.1.tar.gz (122 kB)
Using cached oauthlib-2.0.0.tar.gz (122 kB)
Using cached oauthlib-1.1.2.tar.gz (111 kB)
Using cached oauthlib-1.1.1.tar.gz (108 kB)
Using cached oauthlib-1.1.0-1.tar.gz (106 kB)
WARNING: Discarding https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba (from https://pypi.org/simple/oauthlib/). Requested oauthlib>=0.6.2 from https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tb-nightly~=2.5.0.a->tf-nightly==2.5.0.dev20210316->-r requirements.txt (line 1)) has inconsistent version: filename has '1.1.0.post1', but metadata has '1.1.0'
Using cached oauthlib-1.0.3.tar.gz (109 kB)
Using cached oauthlib-1.0.2.tar.gz (108 kB)
Using cached oauthlib-1.0.1.tar.gz (108 kB)
Using cached oauthlib-1.0.0.tar.gz (108 kB)
Using cached oauthlib-0.7.2.tar.gz (106 kB)
Using cached oauthlib-0.7.1.tar.gz (106 kB)
Using cached oauthlib-0.7.0.tar.gz (105 kB)
Using cached oauthlib-0.6.3.tar.gz (100 kB)
Using cached oauthlib-0.6.2.tar.gz (100 kB)
oli@lucy:~/Downloads/virtual_webcam_background-master$ cat requirements.txt
tf-nightly==2.5.0.dev20210316
numpy==1.19.5
scipy==1.6.1
pillow==8.1.2
pyyaml==5.4.1
pyfakewebcam==0.1.0
opencv-python==4.5.1.48
tfjs-graph-converter==1.4.2
What's the output of pip -V
, @damnms?
Try getting the latest version by running pip install pip==21.0.1
i tried 21.0.1 and 20.3.3, no change, the same behaviour
(venv) oli@lucy:~/Downloads/virtual_webcam_background-master$ pip -V pip 20.3.3 from /usr/local/lib/python3.9/dist-packages/pip (python 3.9)
Yep, I confirm it's back. It's happening to me as well as @damnms:
$ pip --version
pip 21.0.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)
$ pip cache purge
Files removed: 4215
$ pip install tfjs-graph-converter==1.4.2
Collecting tfjs-graph-converter==1.4.2
Downloading tfjs_graph_converter-1.4.2-py3-none-any.whl (32 kB)
Collecting tensorflowjs>=1.5.2
Downloading tensorflowjs-3.3.0-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 1.3 MB/s
Collecting h5py<3,>=2.8.0
Downloading h5py-2.10.0.tar.gz (301 kB)
|████████████████████████████████| 301 kB 4.4 MB/s
Collecting tensorflowjs>=1.5.2
Downloading tensorflowjs-3.2.0-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 2.1 MB/s
Downloading tensorflowjs-3.1.0-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 5.2 MB/s
Downloading tensorflowjs-3.0.0-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 3.6 MB/s
Downloading tensorflowjs-2.8.5-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 5.3 MB/s
Downloading tensorflowjs-2.8.4-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 4.4 MB/s
Downloading tensorflowjs-2.8.3-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 4.7 MB/s
Downloading tensorflowjs-2.8.2-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 5.0 MB/s
Downloading tensorflowjs-2.8.1-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 4.4 MB/s
Downloading tensorflowjs-2.8.0-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 9.5 MB/s
Downloading tensorflowjs-2.7.0-py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 2.9 MB/s
Downloading tensorflowjs-2.6.0-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 1.1 MB/s
Downloading tensorflowjs-2.5.0-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 923 kB/s
Downloading tensorflowjs-2.4.0-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 8.8 MB/s
Collecting numpy<1.19.0,>=1.16.4
Downloading numpy-1.18.5.zip (5.4 MB)
|████████████████████████████████| 5.4 MB 28.2 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
.......
@pradyunsg @uranusjr I though we had this fixed?
The key word of this issue is “eagerly”. At the time, pip would download all versions of a package, only to find that it does not need it after all. That was fixed. In your scenario, pip downloads multiple versions because the versions it downloaded conflict with other packages you specified, which is called backtracking and is a feature. The versions need to be downloaded to detect the conflicts. You are misidentifying the issue.
@uranusjr I think what would greatly help to identify such cases is if pip explained why it is downloading (and discarding) multiple versions then. Even in the most verbose mode, I cannot find any such information, and it's filling hundreds of terminal pages.
Note that I cannot current repro this problem with pip install tfjs-graph-converter==1.4.2
, but pip install tfjs-graph-converter==1.4.2 tf-nightly==2.5.0.dev20210316
still does. And so I would expect
pip install -v -v -v tfjs-graph-converter==1.4.2 tf-nightly==2.5.0.dev20210316 2>&1 | grep "2\.8\.3"
to tell me why tensorflowjs-2.8.3
is considered, but not used. By contrast, everything I get is
Skipping link: none of the wheel's tags match: py2-none-any: https://files.pythonhosted.org/packages/a0/ee/22cefe5a3f6f7e716b88bcd0bbf781d1674a2a61d940fadecfcda6c0c847/tensorflowjs-2.8.3-py2-none-any.whl#sha256=fd186d646dced1f3e21157614ff1f7926c09f019ccf00b5430b873a668991eaf (from https://pypi.org/simple/tensorflowjs/)
Found link https://files.pythonhosted.org/packages/5e/74/60313e6bdcf5d1878ad36a127af073c7b2e2df34abc8030ca7eb1e9a72e7/tensorflowjs-2.8.3-py3-none-any.whl#sha256=fbea42c82d1c8383c546a1ac88b046734585e1e2ee57de18fe69a84bf9a315fe (from https://pypi.org/simple/tensorflowjs/), version: 2.8.3
Looking up "https://files.pythonhosted.org/packages/5e/74/60313e6bdcf5d1878ad36a127af073c7b2e2df34abc8030ca7eb1e9a72e7/tensorflowjs-2.8.3-py3-none-any.whl" in the cache
Using cached tensorflowjs-2.8.3-py3-none-any.whl (63 kB)
Added tensorflowjs>=1.5.2 from https://files.pythonhosted.org/packages/5e/74/60313e6bdcf5d1878ad36a127af073c7b2e2df34abc8030ca7eb1e9a72e7/tensorflowjs-2.8.3-py3-none-any.whl#sha256=fbea42c82d1c8383c546a1ac88b046734585e1e2ee57de18fe69a84bf9a315fe (from tfjs-graph-converter==1.4.2) to build tracker '/tmp/pip-req-tracker-bztc8ovx'
Removed tensorflowjs>=1.5.2 from https://files.pythonhosted.org/packages/5e/74/60313e6bdcf5d1878ad36a127af073c7b2e2df34abc8030ca7eb1e9a72e7/tensorflowjs-2.8.3-py3-none-any.whl#sha256=fbea42c82d1c8383c546a1ac88b046734585e1e2ee57de18fe69a84bf9a315fe (from tfjs-graph-converter==1.4.2) from build tracker '/tmp/pip-req-tracker-bztc8ovx'
Couldn't it say something like
I considered tensorflowjs-2.8.3 but cannot use it because it conflicts with requirement
x>=a.b
(fromy==d.e.f
)?
Here's my output of pip install tfjs-graph-converter==1.4.2 tf-nightly==2.5.0.dev20210316
:
tbh i just want to run this piece of software on debian 11: https://github.com/allo-/virtual_webcam_background
anyone an idea how to get this working when pip behaves like it does? should i create a new report to not spam this one here?
unfortunately we use a lot of conferencing software (zoom, ms teams, etc.) and i dont want anyone be able to see my room etc.
@damnms Probably the issue tracker for the software itself.
Is pip broken again? a pip install -U -r requirements.txt
on production machine is taking several minutes, and querying packages that aren't even in the requirements.txt. I'm hesitant to stop it as I'm not sure what my alternatives are at this point.
I submitted a report to https://pip.pypa.io/surveys/backtracking but that doesn't really help me right now.
Environment
Description If you use pip to update requirements after installing 20.3.2 it goes into an infinite loop resolving and downloading previous versions of dependencies.
This happened on our CI environment that took 26 minutes to complete a job that takes no longer than 3, but then we were able to replicate it locally by just telling pip to update himself...
Expected behavior
Dependencies should be resolved without seeking all versions of a package.
How to Reproduce
pip install -U pip
to get to 20.3.2pip install -U pip
againOutput
First, update pip
Then update it again (I had to ctrl+C to stop the loop)