search

zestsoftware / zest.releaser

Python software releasing made easy and repeatable
https://zestreleaser.readthedocs.io
GNU General Public License v2.0
198 stars 62 forks source link

Verification of registered package fails since pypi.org #281

Closed MasonEgger closed 6 years ago

MasonEgger commented 6 years ago

The check on line 45 of release.py fails due to a failed SSL Cert check. This causes release to no longer upload to http://pypi.org/legacy/

$ release
There is already a tag 1.2.0, show if there are differences? (Y/n)? y
git diff 1.2.0

Check out the tag (for tweaks or pypi/distutils server upload) (Y/n)? y
INFO: Doing a checkout...
Note: checking out '1.2.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b <new-branch-name>

HEAD is now at 88f5661... new version

INFO: Tag checkout placed in /private/var/folders/zf/0mb199fd3zs_0tq75gxg2s3h0000gn/T/meggersample-1.2.0-d85z__40/gitclone
INFO: Changing to sub directory in tag checkout: /private/var/folders/zf/0mb199fd3zs_0tq75gxg2s3h0000gn/T/meggersample-1.2.0-d85z__40/gitclone
INFO: Making a source distribution of a fresh tag checkout (in /private/var/folders/zf/0mb199fd3zs_0tq75gxg2s3h0000gn/T/meggersample-1.2.0-d85z__40/gitclone).
Showing first few lines...
running sdist
running egg_info
creating meggersample.egg-info
writing meggersample.egg-info/PKG-INFO
writing dependency_links to meggersample.egg-info/dependency_links.txt
...
Showing last few lines...
Writing meggersample-1.2.0/setup.cfg
creating dist
Creating tar archive
removing 'meggersample-1.2.0' (and everything under it)

INFO: Making a wheel of a fresh tag checkout (in /private/var/folders/zf/0mb199fd3zs_0tq75gxg2s3h0000gn/T/meggersample-1.2.0-d85z__40/gitclone).
Showing first few lines...
running bdist_wheel
running build
running build_py
creating build
creating build/lib
...
Showing last few lines...
adding 'meggersample-1.2.0.dist-info/WHEEL'
adding 'meggersample-1.2.0.dist-info/METADATA'
adding 'meggersample-1.2.0.dist-info/RECORD'
removing build/bdist.macosx-10.6-intel/wheel

https://pypi.org/simple/meggersample
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Zelgius/.virtualenvs/warehouse/bin/release", line 11, in <module>
    sys.exit(main())
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/release.py", line 341, in main
    releaser.run()
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/baserelease.py", line 391, in run
    self.execute()
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/release.py", line 77, in execute
    self._release()
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/release.py", line 326, in _release
    self._upload_distributions(package)
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/release.py", line 162, in _upload_distributions
    if server == 'pypi' and not package_in_pypi(package):
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/release.py", line 46, in package_in_pypi
    urllib2.urlopen(url)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>
mauritsvanrees commented 6 years ago

I released a package today and that went fine. That was with Python 2.7 though. But this is a lower level problem. I would say there is either something wrong on PyPI (hopefully temporary) or on your system.

I remember that pip 9.0.2 had a fix for Max OS, needed when your openssl is too old. See https://pip.pypa.io/en/stable/news/#id9. Maybe this is something similar. Maybe you need either a newer or an older version of requests. Or a newer openssl library.

Can your browser connect to https://pypi.org? Does pip still work for you? Or using a Python prompt and the same requests version that you use above, check if the following works:

import requests
requests.get('https://pypi.org/')

Theoretically, you could manually edit zest/releaser/release.py so that package_in_pypi returns False without actually doing any checks. But I guess it will then simply go wrong in another line.

MasonEgger commented 6 years ago

I am able to access pypi via requests and through my browser no problem. I believe it may be a problem with the urllib2 found in the six library. The url that is on line 43 is also no longer supported by python and should probably point to https://pypi.org/simple. Maybe the redirection is causing this issue? I am able to use requests to verify that that package is indeed there. I quickly patched the issue with requests here

import requests
...
def package_in_pypi(package):
    """Check whether the package is registered on pypi"""
    url = 'https://pypi.org/simple/%s' % package
    try:
        r = requests.get(url)
        if r.status_code == 200:
            return True
        else:
            return False
    except HTTPError as e:
        logger.debug("Package not found on pypi: %s", e)
        return False

But I'm not sure if this is how you want to move forward. My testing is below

python
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 03:03:55)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://pypi.org/')
<Response [200]>
...
>>> from release import package_in_pypi
>>> package_in_pypi('flask')
https://pypi.org/simple/flask
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/Zelgius/.virtualenvs/warehouse/lib/python3.6/site-packages/zest/releaser/release.py", line 46, in package_in_pypi
    urllib2.urlopen(url)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>
...
...
...
(patch function)
...
...
...
>>> from release import package_in_pypi
>>> package_in_pypi('flask')
True

I can submit a PR with this fix if you wish to move forward with this solution.

mauritsvanrees commented 6 years ago

That is strange. For me it works fine:

>>> requests.get('https://pypi.python.org/simple/flask')
<Response [200]>
>>> from zest.releaser.release import package_in_pypi
>>> package_in_pypi('flask')
True
>>> package_in_pypi('zest.releaser')
True

Tried with Python 2 and 3.

I don't have an explanation for the difference between our machines. But you are right that we should be using the new pypi.org.

I will make a PR. There are a couple more spots where we must change this, though this is the most important one.

mauritsvanrees commented 6 years ago

Fix released in 6.15.0.