urllib certificate errors on MacOS 11.2.3

[jeffcasavant]

I have Python 3.8.10 installed via asdf on my MacOS 11.2.3 machine.

Running the following snippet produces an SSL certificate verification error:

import urllib.request
urllib.request.urlopen("https://google.com")

Obviously Google opens fine in my browser.

[jcasavant@jcasavant-MacBook-Pro ~]$ python
Python 3.8.10 (default, Jul  2 2021, 17:07:24) 
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> urllib.request.urlopen("https://google.com")
Traceback (most recent call last):
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/http/client.py", line 1007, in _send_output
    self.send(msg)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/http/client.py", line 947, in send
    self.connect()
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/http/client.py", line 1421, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/Users/jcasavant/.asdf/installs/python/3.8.10/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)>

The patch described in the readme fails to apply during the build process:

[jcasavant@jcasavant-MacBook-Pro ~]$ export ASDF_PYTHON_PATCH_URL="https://github.com/python/cpython/commit/8ea6353.patch?full_index=1"
[jcasavant@jcasavant-MacBook-Pro ~]$ asdf install python 3.8.10
python-build --patch 3.8.10 /Users/jcasavant/.asdf/installs/python/3.8.10
with patch file from: https://github.com/python/cpython/commit/8ea6353.patch?full_index=1
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.8.10.tar.xz...
-> https://www.python.org/ftp/python/3.8.10/Python-3.8.10.tar.xz
Installing Python-3.8.10...
patching file Misc/NEWS.d/next/macOS/2020-06-24-13-51-57.bpo-41100.mcHdc5.rst
patching file configure
Hunk #1 FAILED at 3426.
1 out of 1 hunk FAILED -- saving rejects to file configure.rej
patching file configure.ac
Hunk #1 FAILED at 510.
1 out of 1 hunk FAILED -- saving rejects to file configure.ac.rej

BUILD FAILED (OS X 11.2.3 using python-build 2.0.2-1-g673c7301)

Inspect or clean up the working tree at /var/folders/hr/lr89gxf13pn47tjg3sz23dmc0000gq/T/python-build.20210702171408.38906
Results logged to /var/folders/hr/lr89gxf13pn47tjg3sz23dmc0000gq/T/python-build.20210702171408.38906.log

Last 10 log lines:
/var/folders/hr/lr89gxf13pn47tjg3sz23dmc0000gq/T/python-build.20210702171408.38906 ~
/var/folders/hr/lr89gxf13pn47tjg3sz23dmc0000gq/T/python-build.20210702171408.38906/Python-3.8.10 /var/folders/hr/lr89gxf13pn47tjg3sz23dmc0000gq/T/python-build.20210702171408.38906 ~

So I'm at a loss. Is this plugin supposed to install some trusted certs on MacOS? I've read that the system / homebrew Python installations sometimes need that done manually.

[jhogendorn]

I'm experiencing this also on 11.4

[jhogendorn]

Quick update, you can move past this by:

pip install certifi
CERT_PATH=$(python -m certifi)
export SSL_CERT_FILE=${CERT_PATH}
export REQUESTS_CA_BUNDLE=${CERT_PATH}