OpenSSL v3 issues

[barronh]

Testing fails on github actions because the defaults of OpenSSL has changed.

/home/runner/work/pyrsig/pyrsig/examples/timeseries/plot_phoenix.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/runner/work/pyrsig/pyrsig/examples/timeseries/plot_phoenix.py", line 22, in <module>
    aqsdf = rsigapi.to_dataframe('aqs.no2', parse_dates=True, unit_keys=False)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pyrsig/pyrsig/pyrsig/__init__.py", line 584, in to_dataframe
    outpath = self.get_file(
              ^^^^^^^^^^^^^^
  File "/home/runner/work/pyrsig/pyrsig/pyrsig/__init__.py", line 410, in get_file
    _getfile(url, outpath, verbose=verbose)
  File "/home/runner/work/pyrsig/pyrsig/pyrsig/__init__.py", line 173, in _getfile
    urlretrieve(
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 241, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
                            ^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line [216](https://github.com/barronh/pyrsig/actions/runs/4451827602/jobs/7818907463#step:6:217), in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:992)>

This appears to be a known issue that is specific to certain government servers.

Unclear right now if this should be addressed in pyrsig or if it should be handled by configuring the users machine. as described on stackoverflow

[hnqtran]

Confirm that the issue can be handled on the user side following the stackoverflow link that Barron provided.

Summary of what I did as example:

1. Create a new conda environment specifically for pyrsig application: $conda env create --name PYRSIG (take note of the environment location)

2. Activate the env: $conda activate PYRSIG

3. Install jupyter notebook (this will install dependent packages and give us the /ssl/openssl.cnf (PYRSIG)$conda install -c conda-forge jupyterlab

4. Remove or backup the /ssl/openssl.cnf. Create new /ssl/openssl.cnf with content:

   openssl_conf = openssl_init

[openssl_init] ssl_conf = ssl_sect

[ssl_sect] system_default = system_default_sect

[system_default_sect] Options = UnsafeLegacyRenegotiation #######

5. Install and use pyrsig as normal: (PYRSIG)$pip install --user netcdf4 pyproj pycno pyrsig

[barronh]

Thanks @boylothoi !

The commands bellow will create a local config (in the present working directory) and adds an environmental variable to use the local file instead of the system version.

printf "openssl_conf = openssl_init\n\n[openssl_init]\nssl_conf = ssl_sect\n\n[ssl_sect]\nsystem_default = system_default_sect\n\n[system_default_sect]\nOptions = UnsafeLegacyRenegotiation\n" > openssl.cnf
export OPENSSL_CONF=${PWD}/openssl.cnf

You can add the environmental variable to your .bashrc to make it persist. To do that, you’d likely want the file in your home folder and the environmental variable would point to it there.

[barronh]

In csh, that would be

printf "openssl_conf = openssl_init\n\n[openssl_init]\nssl_conf = ssl_sect\n\n[ssl_sect]\nsystem_default = system_default_sect\n\n[system_default_sect]\nOptions = UnsafeLegacyRenegotiation\n" > openssl.cnf
setenv OPENSSL_CONF ${PWD}/openssl.cnf

[barronh]

On Google Colab, they have updated so that the OpenSSL now reports the same error. While I don't love it as a long-term solution, it can be solved by:

1. Adding the code below to the top of the notebook
2. Running it.
3. Restarting the kernel
4. Then, running the rest of the notebook.

cnfstr = open('/etc/ssl/openssl.cnf').read().strip()
if not cnfstr.endswith('Options = UnsafeLegacyRenegotiation'):
  cnfstr = cnfstr + '\nOptions = UnsafeLegacyRenegotiation\n'
  open('/etc/ssl/openssl.cnf', 'w').write(cnfstr)
  print('You must restart')
else:
  print('Good configuration')

[barronh]

I am closing this issue because I have added native support for openssl v3.