SSLCertVerificationError on Windows because of OpenSSL configuration

[sven-h]

Hi,

I recently noticed that the download of webpages/APIs using Let's Encrypt does not work on Windows.

Actual Behavior

As an example, the following python code snippet will throw a SSLCertVerificationError on Windows with a recent version of OpenSSL package:

import urllib.request 
with urllib.request.urlopen( 'https://letsencrypt.org' ) as response: 
    response_text = response.read() 

The error (full error at the end) [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1076) is a bit misleading because the certificate is actually valid and not expired. The main problem is that the openSSL package of Anaconda is statically setting the openssldir to %CommonProgramFiles%\ssl. Usually this folder does not exist and thus does not contain any certificates.

You can check which folder it uses by executing openssl version -d after the activation of the corresponding environment.

Due to a change of the Let's Encrypt cerificates, the ISRG Root X1 certificate is not in the trust store anymore (starting from June 2021). Thus the validation of the certificate fails (you can check by executing openssl s_client -connect letsencrypt.org:443 to also see the trace of certificate).

Even tough I copied the certificates from {venv_location}/Library/ssl to %CommonProgramFiles%\ssl the error does not dissapear.

Expected Behavior

Use the certificates from ca-certificates package (which is the case for linux).

Steps to Reproduce

On Windows, create a new venv conda create -n py37 python=3.7 and execute the python snippet from the top.

Operating System: Windows

conda info

``` active environment : py37 active env location : C:\Users\shertlin\Anaconda3\envs\py37 shell level : 2 user config file : C:\Users\shertlin\.condarc populated config files : C:\Users\shertlin\.condarc conda version : 4.9.2 conda-build version : 3.18.11 python version : 3.7.7.final.0 virtual packages : __win=0=0 __archspec=1=x86_64 base environment : C:\Users\shertlin\Anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\Users\shertlin\Anaconda3\pkgs C:\Users\shertlin\.conda\pkgs C:\Users\shertlin\AppData\Local\conda\conda\pkgs envs directories : C:\Users\shertlin\Anaconda3\envs C:\Users\shertlin\.conda\envs C:\Users\shertlin\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/4.9.2 requests/2.25.1 CPython/3.7.7 Windows/10 Windows/10.0.18362 administrator : False netrc file : None offline mode : False ```

conda list --show-channel-urls

``` # packages in environment at C:\Users\shertlin\Anaconda3\envs\py37: # # Name Version Build Channel ca-certificates 2021.7.5 haa95532_1 defaults certifi 2021.5.30 py37haa95532_0 defaults cffi 1.14.6 py37h2bbff1b_0 defaults cryptography 3.4.7 py37h71e12ea_0 defaults openssl 1.1.1l h2bbff1b_0 defaults pip 21.0.1 py37haa95532_0 defaults pycparser 2.20 py_2 defaults pyopenssl 20.0.1 pyhd3eb1b0_1 defaults python 3.7.11 h6244533_0 defaults setuptools 58.0.4 py37haa95532_0 defaults six 1.16.0 pyhd3eb1b0_0 defaults sqlite 3.36.0 h2bbff1b_0 defaults vc 14.2 h21ff451_1 defaults vs2015_runtime 14.27.29016 h5e58377_2 defaults wheel 0.37.0 pyhd3eb1b0_1 defaults wincertstore 0.2 py37haa95532_2 defaults ```

Full error:

---------------------------------------------------------------------------
SSLCertVerificationError                  Traceback (most recent call last)
~\Anaconda3\lib\urllib\request.py in do_open(self, http_class, req, **http_conn_args)
   1318                 h.request(req.get_method(), req.selector, req.data, headers,
-> 1319                           encode_chunked=req.has_header('Transfer-encoding'))
   1320             except OSError as err: # timeout error

~\Anaconda3\lib\http\client.py in request(self, method, url, body, headers, encode_chunked)
   1251         """Send a complete request to the server."""
-> 1252         self._send_request(method, url, body, headers, encode_chunked)
   1253 

~\Anaconda3\lib\http\client.py in _send_request(self, method, url, body, headers, encode_chunked)
   1297             body = _encode(body, 'body')
-> 1298         self.endheaders(body, encode_chunked=encode_chunked)
   1299 

~\Anaconda3\lib\http\client.py in endheaders(self, message_body, encode_chunked)
   1246             raise CannotSendHeader()
-> 1247         self._send_output(message_body, encode_chunked=encode_chunked)
   1248 

~\Anaconda3\lib\http\client.py in _send_output(self, message_body, encode_chunked)
   1025         del self._buffer[:]
-> 1026         self.send(msg)
   1027 

~\Anaconda3\lib\http\client.py in send(self, data)
    965             if self.auto_open:
--> 966                 self.connect()
    967             else:

~\Anaconda3\lib\http\client.py in connect(self)
   1421             self.sock = self._context.wrap_socket(self.sock,
-> 1422                                                   server_hostname=server_hostname)
   1423 

~\Anaconda3\lib\ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    422             context=self,
--> 423             session=session
    424         )

~\Anaconda3\lib\ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
    869                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 870                     self.do_handshake()
    871             except (OSError, ValueError):

~\Anaconda3\lib\ssl.py in do_handshake(self, block)
   1138                 self.settimeout(None)
-> 1139             self._sslobj.do_handshake()
   1140         finally:

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1076)

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
<ipython-input-72-187704f837db> in <module>
      1 import urllib.request
----> 2 with urllib.request.urlopen( 'https://letsencrypt.org' ) as response:
      3 #with urllib.request.urlopen( 'https://google.com:443' ) as response:
      4     response_text = response.read()

~\Anaconda3\lib\urllib\request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    220     else:
    221         opener = _opener
--> 222     return opener.open(url, data, timeout)
    223 
    224 def install_opener(opener):

~\Anaconda3\lib\urllib\request.py in open(self, fullurl, data, timeout)
    523             req = meth(req)
    524 
--> 525         response = self._open(req, data)
    526 
    527         # post-process response

~\Anaconda3\lib\urllib\request.py in _open(self, req, data)
    541         protocol = req.type
    542         result = self._call_chain(self.handle_open, protocol, protocol +
--> 543                                   '_open', req)
    544         if result:
    545             return result

~\Anaconda3\lib\urllib\request.py in _call_chain(self, chain, kind, meth_name, *args)
    501         for handler in handlers:
    502             func = getattr(handler, meth_name)
--> 503             result = func(*args)
    504             if result is not None:
    505                 return result

~\Anaconda3\lib\urllib\request.py in https_open(self, req)
   1360         def https_open(self, req):
   1361             return self.do_open(http.client.HTTPSConnection, req,
-> 1362                 context=self._context, check_hostname=self._check_hostname)
   1363 
   1364         https_request = AbstractHTTPHandler.do_request_

~\Anaconda3\lib\urllib\request.py in do_open(self, http_class, req, **http_conn_args)
   1319                           encode_chunked=req.has_header('Transfer-encoding'))
   1320             except OSError as err: # timeout error
-> 1321                 raise URLError(err)
   1322             r = h.getresponse()
   1323         except:

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1076)>

[cmosguy]

@sven-h did you figure out any workaround for this?

[sven-h]

Hi @cmosguy , unfortunately not. Do you experienced the same problem?

[cmosguy]

I gave up @sven-h , dumping this package...

[Ndron]

Try:

import ssl
ssl._create_default_https_context = ssl._create_unverified_context