Closed Duoquote closed 1 year ago
Looks like the issue is about this specific package called connector-x
, feels like the file is corrupted maybe?
Python 3.9.13 pip 22.3 from C:\Users\duoqu\miniconda3\envs\tf\lib\site-packages\pip (python 3.9)
Collecting connectorx
Downloading connectorx-0.3.1-cp39-none-win_amd64.whl (41.3 MB)
---------------------------------------- 41.3/41.3 MB 8.7 MB/s eta 0:00:00
Installing collected packages: connectorx
Successfully installed connectorx-0.3.1
Python 3.8.5 pip 22.1.2 from C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip (python 3.8)
Collecting connectorx
Downloading connectorx-0.3.1-cp38-none-win_amd64.whl (41.3 MB)
---- ----------------------------------- 4.2/41.3 MB 5.2 MB/s eta 0:00:08
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\urllib3\response.py", line 435, in _error_catcher
yield
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\urllib3\response.py", line 516, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Users\duoqu\miniconda3\envs\junk\lib\http\client.py", line 458, in read
n = self.readinto(b)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\http\client.py", line 502, in readinto
n = self.fp.readinto(b)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper
return func(self, options, args)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\commands\install.py", line 341, in run
requirement_set = resolver.resolve(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 94, in resolve
result = self._result = resolver.resolve(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 481, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 348, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 172, in _add_to_criteria
if not criterion.candidates:
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 151, in __bool__
return bool(self._sequence)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 155, in __bool__
return any(self)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 143, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 47, in _iter_built
candidate = func()
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 215, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 291, in __init__
super().__init__(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 161, in __init__
self.dist = self._prepare()
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 230, in _prepare
dist = self._prepare_distribution()
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 302, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\operations\prepare.py", line 428, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\operations\prepare.py", line 473, in _prepare_linked_requirement
local_file = unpack_url(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\operations\prepare.py", line 155, in unpack_url
file = get_http_url(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\operations\prepare.py", line 96, in get_http_url
from_path, content_type = download(link, temp_dir.path)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\network\download.py", line 146, in __call__
for chunk in chunks:
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\cli\progress_bars.py", line 53, in _rich_progress_bar
for chunk in iterable:
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_internal\network\utils.py", line 63, in response_chunks
for chunk in response.raw.stream(
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\urllib3\response.py", line 573, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\urllib3\response.py", line 538, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "C:\Users\duoqu\miniconda3\envs\junk\lib\site-packages\pip\_vendor\urllib3\response.py", line 440, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
We have the same error, but for multiple packages, including spacy==2.3.8
, pex==2.1.102
and pantsbuild.pants==2.13.0
. To me this looks rather like a regional network or availability issue than something specifically related to the packages.
I have a similar issue downloading simplemma==0.8.2
(79MB), simplemma==0.9.0
(90MB) and fasttext_wheel==0.9.2
(4.4MB) wheels from several locations in Finland; IPv4 vs IPv6 makes no difference. The download gets stuck in the middle. pip downloads quickly time out, but if I use wget --continue
directly on the download URL, the download will first proceed smoothly to some point (e.g. 50%), then it will freeze and proceed only very very slowly.
When I do this simultaneously from two different machines (in different networks), it seems to me that both will advance at the same rate. When one gets to 78%, the other one will soon reach 78% as well. I don't know much about how the CDN works, but it seems to me that it could be related to the CDN setup.
I have the same issue (read time out on files.pythonhosted.org) for some packages, but not all packages. Here, it fails on cupy-cuda112==9.6.0. If I remove that package from my requirements.txt, it fails on another package somewhere else down the line (differs each time). I've seen failures on ngt==1.12.2 and Django==2.1.14. I am in New Jersey, USA.
[ 7/14] RUN pip3 install --upgrade pip setuptools wheel && pip3 install -r requirements.txt:
#0 0.917 Collecting pip
#0 1.089 Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)
#0 1.939 Collecting setuptools
#0 2.351 Downloading https://files.pythonhosted.org/packages/b0/3a/88b210db68e56854d0bcf4b38e165e03be377e13907746f825790f3df5bf/setuptools-59.6.0-py3-none-any.whl (952kB)
#0 2.552 Collecting wheel
#0 2.757 Downloading https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl
#0 2.780 Installing collected packages: pip, setuptools, wheel
#0 2.780 Found existing installation: pip 9.0.3
#0 2.962 Uninstalling pip-9.0.3:
#0 4.019 Successfully uninstalled pip-9.0.3
#0 4.998 Found existing installation: setuptools 39.2.0
#0 5.060 Uninstalling setuptools-39.2.0:
#0 5.409 Successfully uninstalled setuptools-39.2.0
#0 5.816 Successfully installed pip-21.3.1 setuptools-59.6.0 wheel-0.37.1
#0 6.118 You are using pip version 21.3.1, however version 22.3.1 is available.
#0 6.118 You should consider upgrading via the 'pip install --upgrade pip' command.
#0 6.675 Requirement already satisfied: wheel==0.37.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (0.37.1)
#0 6.676 Requirement already satisfied: pip==21.3.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 2)) (21.3.1)
#0 6.908 Collecting pyinstaller==4.10
#0 6.983 Downloading pyinstaller-4.10-py3-none-manylinux2014_x86_64.whl (1.5 MB)
#0 7.101 Collecting altgraph==0.17.2
#0 7.115 Downloading altgraph-0.17.2-py2.py3-none-any.whl (21 kB)
#0 7.185 Collecting asgiref==3.4.1
#0 7.199 Downloading asgiref-3.4.1-py3-none-any.whl (25 kB)
#0 7.260 Collecting certifi==2021.5.30
#0 7.275 Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
#0 7.460 Collecting charset-normalizer==2.0.3
#0 7.474 Downloading charset_normalizer-2.0.3-py3-none-any.whl (35 kB)
#0 7.692 Collecting cityhash==0.4.3
#0 7.708 Downloading cityhash-0.4.3-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (483 kB)
#0 7.792 Collecting click==8.0.1
#0 7.808 Downloading click-8.0.1-py3-none-any.whl (97 kB)
#0 7.884 Collecting cupy-cuda112==9.6.0
#0 7.948 Downloading cupy_cuda112-9.6.0-cp36-cp36m-manylinux1_x86_64.whl (75.9 MB)
#0 23.74 ERROR: Exception:
#0 23.74 Traceback (most recent call last):
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 438, in _error_catcher
#0 23.74 yield
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 519, in read
#0 23.74 data = self._fp.read(amt) if not fp_closed else b""
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read
#0 23.74 data = self.__fp.read(amt)
#0 23.74 File "/usr/lib64/python3.6/http/client.py", line 459, in read
#0 23.74 n = self.readinto(b)
#0 23.74 File "/usr/lib64/python3.6/http/client.py", line 503, in readinto
#0 23.74 n = self.fp.readinto(b)
#0 23.74 File "/usr/lib64/python3.6/socket.py", line 586, in readinto
#0 23.74 return self._sock.recv_into(b)
#0 23.74 File "/usr/lib64/python3.6/ssl.py", line 971, in recv_into
#0 23.74 return self.read(nbytes, buffer)
#0 23.74 File "/usr/lib64/python3.6/ssl.py", line 833, in read
#0 23.74 return self._sslobj.read(len, buffer)
#0 23.74 File "/usr/lib64/python3.6/ssl.py", line 590, in read
#0 23.74 v = self._sslobj.read(len, buffer)
#0 23.74 socket.timeout: The read operation timed out
#0 23.74
#0 23.74 During handling of the above exception, another exception occurred:
#0 23.74
#0 23.74 Traceback (most recent call last):
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
#0 23.74 status = run_func(*args)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
#0 23.74 return func(self, options, args)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/commands/install.py", line 339, in run
#0 23.74 reqs, check_supported_wheels=not options.target_dir
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 93, in resolve
#0 23.74 collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 482, in resolve
#0 23.74 state = resolution.resolve(requirements, max_rounds=max_rounds)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 349, in resolve
#0 23.74 self._add_to_criteria(self.state.criteria, r, parent=None)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
#0 23.74 if not criterion.candidates:
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
#0 23.74 return bool(self._sequence)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
#0 23.74 return any(self)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
#0 23.74 return (c for c in iterator if id(c) not in self._incompatible_ids)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
#0 23.74 candidate = func()
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
#0 23.74 version=version,
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 287, in __init__
#0 23.74 version=version,
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
#0 23.74 self.dist = self._prepare()
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
#0 23.74 dist = self._prepare_distribution()
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
#0 23.74 return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
#0 23.74 return self._prepare_linked_requirement(req, parallel_builds)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 528, in _prepare_linked_requirement
#0 23.74 link, req.source_dir, self._download, self.download_dir, hashes
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 217, in unpack_url
#0 23.74 hashes=hashes,
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 94, in get_http_url
#0 23.74 from_path, content_type = download(link, temp_dir.path)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/network/download.py", line 145, in __call__
#0 23.74 for chunk in chunks:
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/cli/progress_bars.py", line 144, in iter
#0 23.74 for x in it:
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_internal/network/utils.py", line 87, in response_chunks
#0 23.74 decode_content=False,
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 576, in stream
#0 23.74 data = self.read(amt=amt, decode_content=decode_content)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 541, in read
#0 23.74 raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
#0 23.74 File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
#0 23.74 self.gen.throw(type, value, traceback)
#0 23.74 File "/usr/src/app/venv/lib64/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher
#0 23.74 raise ReadTimeoutError(self._pool, None, "Read timed out.")
#0 23.74 pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
------
failed to solve: executor failed running [/bin/sh -c pip3 install --upgrade pip setuptools wheel && pip3 install -r requirements.txt]: exit code: 2
make: *** [container] Error 17
Tracking this incident here: https://status.python.org/incidents/j369h2rkqsgz
We've opened a support request with our CDN provider. As it seems to primarily be impacting bigger files, it likely is to do with configurations supporting Segmented Caching for larger files. We added support for this in December. Ref https://github.com/pypi/infra/pull/112, https://github.com/pypi/infra/pull/114, and https://github.com/pypi/infra/pull/115
We're continuing to investigate with our CDN, but found a potentially related outage published by our backend that we are now tracking as well.
We're seeing improvements after reports from our backend providers that their incident is resolved.
Anyone able to reproduce at this time?
I have successfully installed over 100 packages including torch
(887MB) without any timeouts, the issue looks like solved on my side.
Thanks @ewdurbin ! I see that this has now been marked as resolved in the incident report, and when I tested downloading the simplemma wheel again, the download worked fine.
My Platform
pip3 version:
pip 22.3.1 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)
Full command output:
Fastly Debug
No response
DNS Resolution
Traceroutes / IPv4
Same applies for my local network and another server.
Traceroutes / IPv6 (If available)
No response
HTTPS Requests / IPv4
HTTPS Requests / IPv6 (If available)
No response
TLS Debug / IPv4
TLS Debug / IPv6 (If available)
Code of Conduct