OpenSSL 3 ports migration

[fichtner]

Important notices

Our forum is located at https://forum.opnsense.org , please consider joining discussions there in stead of using GitHub for these matters.

Before you ask a new question, we ask you kindly to acknowledge the following:

• [x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• [x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Collection of random things to take care of:

dns/ddclient (native):

root@OPNsense:~ # /usr/local/opnsense/scripts/ddclient/ddclient_opn.py -l
Traceback (most recent call last):
  File "/usr/local/opnsense/scripts/ddclient/ddclient_opn.py", line 46, in <module>
    print(json.dumps(AccountFactory().known_services()))
  File "/usr/local/opnsense/scripts/ddclient/lib/poller.py", line 41, in __init__
    self._register()
  File "/usr/local/opnsense/scripts/ddclient/lib/poller.py", line 50, in _register
    importlib.import_module(".%s" % os.path.splitext(os.path.basename(filename))[0], pkg_name)
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/opnsense/scripts/ddclient/lib/account/aws.py", line 33, in <module>
    import boto3
  File "/usr/local/lib/python3.9/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/usr/local/lib/python3.9/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/usr/local/lib/python3.9/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/usr/local/lib/python3.9/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/usr/local/lib/python3.9/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/usr/local/lib/python3.9/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import (
  File "/usr/local/lib/python3.9/site-packages/botocore/docs/client.py", line 18, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/usr/local/lib/python3.9/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/usr/local/lib/python3.9/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/usr/local/lib/python3.9/site-packages/botocore/utils.py", line 37, in <module>
    import botocore.httpsession
  File "/usr/local/lib/python3.9/site-packages/botocore/httpsession.py", line 45, in <module>
    from urllib3.contrib.pyopenssl import (
  File "/usr/local/lib/python3.9/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
    import OpenSSL.crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import SSL, crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/SSL.py", line 9, in <module>
    from OpenSSL._util import (
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 167, in <module>
    Binding.init_static_locks()
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 134, in init_static_locks
    cls._ensure_ffi_initialized()
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 123, in _ensure_ffi_initialized
    _legacy_provider_error(cls._legacy_provider_loaded)
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 43, in _legacy_provider_error
    raise RuntimeError(
RuntimeError: OpenSSL 3.0's legacy provider failed to load. This is a fatal error by default, but cryptography supports running without legacy algorithms by setting the environment variable CRYPTOGRAPHY_OPENSSL_NO_LEGACY. If you did not expect this error, you have likely made a mistake with your OpenSSL configuration.

[AdSchellevis]

if it's only for our ddclient alternative, it make be an idea to see if export CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1 would fix it. (https://cryptography.io/en/latest/openssl/#legacy-provider-in-openssl-3-x)

[fichtner]

@AdSchellevis the same actually applies to aliases since it's all Python scripts... https://forum.opnsense.org/index.php?topic=37108.0 so I guess that explains the behaviour recently reported about OpenSSL 3/alias combo.

[AdSchellevis]

yep, it expects https://www.openssl.org/docs/man3.0/man7/OSSL_PROVIDER-legacy.html to support older algorithms, probably an openssl build flag

[fichtner]

So this https://github.com/opnsense/tools/commit/57711c6b ?! So now this isn't a Python issue, it's a FreeBSD ports defaults issue??? -.-

[AdSchellevis]

I'm afraid so, yes

[AdSchellevis]

(missed it the first time as well, but reading a bit deeper indeed points into a different direction)

[fichtner]

at least from my perspective that is the ideal fix for the time being. trying to confirm now but the nightly download is a bit slow at the moment

[fichtner]

Voiced concern over FreeBSD ports handling here https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273656 ... maybe py-cryptography is the right place to bring a "fix" in for everyone else but I'd think we cannot go without legacy for a while anyway.

Confirmed fixed via https://forum.opnsense.org/index.php?topic=37108.msg181665#msg181665 (original reporter of ddclient native issue).