search

zestsoftware / zest.releaser

Python software releasing made easy and repeatable
https://zestreleaser.readthedocs.io
GNU General Public License v2.0
199 stars 62 forks source link

Upload to PyPI fails #414

Closed icemac closed 1 year ago

icemac commented 1 year ago

Using zest.releaser 8.0.0 on Python 3.11 (twine==4.0.2)

I get the following traceback while releasing to PyPI:

...
INFO: Making a wheel of a fresh tag checkout (in /private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/zope.size-5.0-_61cp4yu/gitclone).
Showing first few lines...
running bdist_wheel
running build
running build_py
creating build
creating build/lib
...
Showing last few lines...
adding 'zope.size-5.0.dist-info/namespace_packages.txt'
adding 'zope.size-5.0.dist-info/top_level.txt'
adding 'zope.size-5.0.dist-info/RECORD'
removing build/bdist.macosx-13.0-x86_64/wheel

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mac/.local/bin/fullrelease", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/fullrelease.py", line 27, in main
    releaser.run()
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/baserelease.py", line 433, in run
    self.execute()
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/release.py", line 86, in execute
    self._release()
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/release.py", line 322, in _release
    self._upload_distributions(package)
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/release.py", line 181, in _upload_distributions
    if not self._ask_upload(package, server, register):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/release.py", line 200, in _ask_upload
    if server == "pypi" and not package_in_pypi(package):
                                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mac/.local/pipx/venvs/zest-releaser/lib/python3.11/site-packages/zest/releaser/release.py", line 54, in package_in_pypi
    request.urlopen(url)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 557, in error
    result = self._call_chain(*args)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 749, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 557, in error
    result = self._call_chain(*args)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 749, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 54] Connection reset by peer>

It seems package_in_pypi() calls a URL PyPI does not like. Uploading directly using twine from the temporary checkout directory works fine.

icemac commented 1 year ago

Looking into what package_in_pypi() does, I tried to reproduce:

$ python3.11
Python 3.11.4 (main, Jun 10 2023, 10:30:46) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib import request
>>> package = 'zope.size'
>>> url = "https://pypi.org/simple/%s" % package
>>> request.urlopen(url)
<http.client.HTTPResponse object at 0x10ade1fc0>
>>> url = "https://pypi.org/simple/%s/" % package
>>> request.urlopen(url)
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 54] Connection reset by peer>

When trying directly in Python I get an exception when the URL ends in a / but package_in_pypi() does not seem to do this. It looks a bit strange to me.

reinout commented 1 year ago

It might be the way the dot in the namespace package's name is handled. https://pypi.org/simple/zope.size (dot and no slash) is being 301-redirected to https://pypi.org/simple/zope.size/ (dot and slash) which in turn is being 301-redirected to https://pypi.org/simple/zope-size/ (dash and slash).

I don't remember seeing it that way: with the dash. Why it would lead to the error you're seeing: I don't know yet.

reinout commented 1 year ago

Could it just have been a connection issue? If I try the urlopen now, I don't get a "connection reset by peer":

$ python3
Python 3.11.4 (main, Jun 15 2023, 07:55:38) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib import request
>>> request.urlopen("https://pypi.org/simple/zest.releaser")
<http.client.HTTPResponse object at 0x1013e5ff0>
>>> request.urlopen("https://pypi.org/simple/zest.releaser/")
<http.client.HTTPResponse object at 0x1013e5ed0>
>>> request.urlopen("https://pypi.org/simple/zest-releaser/")
<http.client.HTTPResponse object at 0x1007a2890>
icemac commented 1 year ago

It was hopefully a connection issue. The question would be if zest.releaser could handle it instead of presenting a traceback? I've seen this at two days in the last weeks. Maybe it will occur again in the future.

reinout commented 1 year ago

A try/except should be possible. I'm not going to do it myself as I hope it is a really temporary and rare issue. Pypi is behind a CDN, so it should normally be quite perfect.

Note: https://github.com/pypi/support/issues/new?assignees=&labels=network&template=access-issues.yml has some tips to debug such an issue. Some ipv4/ipv6 curl stuff and so.

icemac commented 1 year ago

Okay, then let's keep it as it is and hope for the best in future.