search

pypi / legacy

This PyPI is no more! See https://github.com/pypa/warehouse.
Other
61 stars 46 forks source link

"499 Client Error: Client Disconnected for url: https://upload.pypi.org/legacy/" #675

Closed s-m-e closed 6 years ago

s-m-e commented 7 years ago

I filed a detailed bug against twine but eventually learned that it also occurs when using setuptools (setup.py ... upload). So it might be an issue in pypi-legacy instead (?).

In a nutshell: I want to upload a (new) package to pypi after having successfully tested it (many times) with pypitest with identical configuration parameters. Uploading of files to pypi fails with HTTP error 499 (with both, twine and setuptools).

My .pypirc file looks like this:

[distutils]
index-servers =
  pypi
  pypitest

[pypi]
username=username
password=password

[pypitest]
repository=https://test.pypi.org/legacy/
username=username
password=password

I am using Python 3.6.1 on Linux x86_64 with twine (1.9.1), requests (2.18.2), requests-toolbelt (0.8.0), urllib3 (1.22) and setuptools (36.2.2).

Even without the repository=... parameter in .pypirc, twine defaults to https://upload.pypi.org/legacy/ when trying to upload to pypi. Fun part: The behavior does not change at all if I provide false login credentials like a wrong password. Logging into pypi via the website works for me, however.

As far as I can tell, HTTP 499 is an error code specific to nginx indicating that the client "just" disconnected. Is this a timeout issue due to server load on pypi or did I misconfigure .pypirc in some sort of way (masquerading a log-in issue for instance) or is this a bug?

I have also tried to follow outdated instructions and attempted to register my files / project first - no change.

EDIT (1): I hacked twine a bit to get more detailed debugging output by patching the _upload routine in twine/repository.py. Now it shows me the full server response, i.e. resp.text / resp.content. Turns out to be interesting. Error 499 is accompanied by an HTML page: 

('<!DOCTYPE html>\n'                                                                                                                                  
 '\t<html>\n'
 '\t  <head>\n'
 '\t\t<meta name="viewport" content="width=device-width, initial-scale=1">\n'
 '\t\t<meta charset="utf-8">\n'
 '\t\t<title>Application Error</title>\n'
 '\t\t<style media="screen">\n'
 '\t\t  html,body,iframe {\n'
 '\t\t\tmargin: 0;\n'
 '\t\t\tpadding: 0;\n'
 '\t\t  }\n'
 '\t\t  html,body {\n'
 '\t\t\theight: 100%;\n'
 '\t\t\toverflow: hidden;\n'
 '\t\t  }\n'
 '\t\t  iframe {\n'
 '\t\t\twidth: 100%;\n'
 '\t\t\theight: 100%;\n'
 '\t\t\tborder: 0;\n'
 '\t\t  }\n'
 '\t\t</style>\n'
 '\t  </head>\n'
 '\t  <body>\n'
 '\t\t<iframe '
 'src="//www.herokucdn.com/error-pages/application-error.html"></iframe>\n'
 '\t  </body>\n'
 '\t</html>')

The page in the iframe says "An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details. " ... My pypi username is "smernst", the package I am trying to upload is named "zugbruecke".

s-m-e commented 7 years ago

After a lot of tests I learned the following:

So, as a (hopefully temporary) workaround I can only recommend the following:

# first shell
ssh -D 8080 some_server_instance_in_aws
# second shell
pip install -vU requests[socks]
export http_proxy=socks5://127.0.0.1:8080 https_proxy=socks5://127.0.0.1:8080
twine upload $filename $filename.asc

The first command connects to my AWS instance via SSH and uses it as a SOCKS proxy on port 8080. You can run this in the background, but I prefer to run it in a separate shell so I see what is going on. The second command (in a new shell) adds SOCKS support to requests. The third one changes the _httpproxy environment variable telling programs about the proxy (twine will honor it) and the fourth one just uploads my package as expected. I only need to do this for pypi, not for pypitest.

boazmohar commented 7 years ago

Have the same problem. Could not verify that re-routing works ( I don't have a AWS instance). Would be happy for any other recommendations. Recreated in Windows, Mac and Unix. Again a wrong password in pypirc has no effect.

s-m-e commented 7 years ago

@boazmohar I had a few chats on freenode#pypa (IRC) with pypi people - the authentication happens after the upload. Since the upload fails, wrong login credentials have no effect. From where / what kind of network / ISP are you trying to upload? Seems like this issue is not an isolated case if you're trying to do this from the other side of the Atlantic. My ISP is Germany's T-Online.

(Maybe you have access to a university computer or proxy you could use for re-routing traffic through it?)

boazmohar commented 7 years ago

@s-m-e I can verify that using my phone as a wifi hotspot I can upload to pypi. My institute is HHMI Janelia research campus. I filed a help-desk ticket for them to unblock the traffic. I will report back if that was indeed the cause.

k0rmarun commented 7 years ago

Same over here. This issue appears on :

and in the networks of:

Could not verify Amazon trick due to missing instance

Faeriol commented 7 years ago

For those of you with this issue... Can you check what version of requests you are using? This didn't work for me until I downgraded from 2.18.4 to 2.14.2.

Edit: If we get a few confirmations, I might just dig into the issue for fun Edit 2: Now that I've got this working on any box, I can't seem to reproduce on any other... Strange

coldfix commented 6 years ago

@s-m-e I have the same issue, from unitymedia network south germany (Heidelberg). I can verify that proxying via my vserver as by your description solves the issue!

Possibly any proxy into a different network may solve the issue.

coldfix commented 6 years ago

@Faeriol downgrading requests did not work for me.

ewdurbin commented 6 years ago

uploads are no longer accepted by the pypi-legacy codebase.

further discussion of this issue should be filed with https://github.com/pypa/warehouse/issues

s-m-e commented 6 years ago

@ewdurbin While I appreciate an answer from someone working on PyPI, I'd like to suggest to update the documentation (in a number of places) of how to upload a Python package, where to file bugs (for certain "sub systems" of PyPI) and what the current state of affairs is with regards to "pypi-legacy" and "warehouse". I gathered my info with respect to the current upload process and this bug from all sorts of places, including podcasts (interviews) and irc chats. It is by no means obvious to the outside world, which piece of code is actually responsible for accepting uploads to PyPI and where to file bugs. There is a new website, and an old one, there is a new "pre-production" backend, and there is an old one, there are different upload URLs (not) doing different things ... After all, pypi.org still has a warning at its top ("This is a pre-production deployment of Warehouse"). I suggest to replace it with a line telling people that you need help with updating your documentation and a link to where interested folks should go first ;)

ewdurbin commented 6 years ago

@s-m-e thanks for the wall of text.

ewdurbin commented 6 years ago

@s-m-e Trying to break down your larger concerns:

Overall, this is no longer to do with the 499 issue and if you'd like to continue to discuss checkout https://github.com/pypa/packaging-problems