dimbleby
commented
5 months ago TimeoutError
The read operation timed out
ReadTimeoutError
ConnectionError
looks like a network connectivity problem, not something that can be solved from this repository.
Open damonallison opened 5 months ago
dimbleby
commented
5 months ago TimeoutError
The read operation timed out
ReadTimeoutError
ConnectionError
looks like a network connectivity problem, not something that can be solved from this repository.
damonallison
commented
5 months ago Hey @dimbleby - thanks for looking into this..
I agree it definitely looks like a network connectivity problem. I can assure you that switching to 1.7.1 and doing the same operation succeeds under the same network conditions, where 1.8.2 will consistently fail.
I think the problem is related to jfrog's mis-handling of ranges as described in #9056.
I will put some time into debugging jfrog's responses and determine why poetry is timing out on the request. My guess is it's waiting for a chunk that it never receives.
dimbleby
commented
5 months ago I don't know whether it is making any difference or not, but you shouldn't put this in your pyproject.toml
[[tool.poetry.source]]
# Please use drone artifactory-poetry-publish step to publish packages.
name = "myco-deploy"
url = "https://artifactory.gcp.mycotech.com/artifactory/api/pypi/pypi-local"
priority = "explicit"publishable repositories are poetry configuration, not project configuration - https://python-poetry.org/docs/repositories/#publishable-repositories
possibly having two repositories with the same domain is confusing things, or possibly not
damonallison
commented
5 months ago Thanks again @dimbleby - appreciate the tip.. I removed the publish repo from pyproject.toml and verified the download failure still exists.
Also of note: I ran poetry install with 1.7.1 to load the cache, then removed the environment and ran poetry install with 1.8.2 - the install ran fine as it didn't have to talk with artifactory.
So it appears like this was introduced between 1.7 and 1.8. I will still plan on debugging the root cause as I believe it's jfrog / artifactory not handling ranges correctly.
dimbleby
commented
5 months ago this bit of output
Source (myco-resolve): Disabling lazy wheel support for artifactory.gcp.mycotech.com: did not receive partial content: got code 200
suggests that poetry has understood that whatever it is talking to does not understand ranges and has given up on using them. But of course it is possible for that code to be bugged
you could try setting solver.lazy-wheel = false to be surer.
damonallison
commented
5 months ago @dimbleby thanks again for the tip... Setting solver.lazy-wheel to false actually works around the problem as well.
damonallison
commented
5 months ago Repro steps:
poetry updatepoetry update
jack-mcivor
commented
5 months ago FWIW I am seeing the same issue on a different private PyPI, also hosted on JFrog artifactory. Poetry v1.8.2
Source (<>): Downloading: ...
[urllib3:urllib3.connectionpool] [https://<>:443](https://<>/) "GET ... HTTP/1.1" 200 16531752
Source (<>): Abort downloading ... because server supports range requests
begin fetching content length
initial bytes request: bytes=-10000
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (3): <>:443
[urllib3:urllib3.connectionpool] [https://<>:443](https://<>/) "GET ... HTTP/1.1" 200 16531752
Source (<>): Disabling lazy wheel support for <>: did not receive partial content: got code 200
Source (<>): Downloading: ...
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (4): <>:443
[urllib3:urllib3.connectionpool] [https://<>:443](https://<>/) "GET ... HTTP/1.1" 200 16531752poetry config solver.lazy-wheel false also solves it for me
dimbleby
commented
5 months ago someone who is hitting this is gonna have to do some debugging themselves I'm afraid. The repro at last-comment-but-one just says that lazy wheel is broken - which obviously is not generally the case.
presumably artifactory is doing something odd in response to range requests and poetry is not able to cope with it: but it's going to take someone with access to such a server to investigate
jack-mcivor
commented
5 months ago I have access to such a server and can help with some guidance. In my case, it seems to be simply ignoring the Range request header and returning a 200 with the full content. This is despite a Accept-Ranges: bytes response header
radoering
commented
5 months ago The behavior in https://github.com/python-poetry/poetry/issues/9185#issuecomment-2047645186 is expected and does not show a final failure but just correct error handling:
The error handling is triggered by Artifactory's presumably bad behavior described in https://github.com/python-poetry/poetry/issues/9068#issuecomment-1973535725. The error handling results in a higher number of network requests, which may overload Artifactory so it times out. 🤷
Currently, we will try range requests again for each wheel (because there are servers that support ranges requests for some wheels and can be trusted regarding the Accept-Ranges header). We could try to distinguish between "good server that supports range requests for some files" (-> try range request for each file the server says it accepts ranges) and "bad server that pretends to support range requests but does not" (do never try range requests again) - or you may just turn off lazy-wheel if you are using a "bad server".
damonallison
commented
5 months ago I also have access to a "bad server" (jfrog artifactory) and can help with some guidance. I'll take an initial stab at trying to distinguish between "good" and "bad" servers and turning off range requests for bad servers.
dimbleby
commented
5 months ago @SergeyTsaplin please dont hijack issues with something that you know to be different. What you are seeing has getting on for a dozen duplicates by now, but this is not one of them.
Description
We are running into errors downloading from a private virtual artifactory instance. It appears almost directly related or the same as #9056 - but since that issue looks resolved and released with
1.8.2, I thought I'd report it again.Workarounds
Use
1.7.1or lowerPoetry Installation Method
pipx
Operating System
macOS 14.4
Poetry Version
Poetry (version 1.8.2)
Poetry Configuration
Python Sysconfig
Example pyproject.toml
Poetry Runtime Logs