oscrypto segfaults on catalina

[worldwise001]

I've come here after an all-day debugging session. tl;dr I think oscrypto is picking up the wrong SSL backend for catalina (OS X 10.15.1) which causes a crash.

There's some workarounds as documented here: https://forums.developer.apple.com/thread/119429 Though that seems pretty gnarly.

Specifically I'm finding it failing in _libcrypto_cffi:

libcrypto_path = _backend_config().get('libcrypto_path')
if libcrypto_path is None:
    libcrypto_path = find_library('crypto')
if not libcrypto_path:
    raise LibraryNotFoundError('The library libcrypto could not be found')

try:
    vffi = FFI()
    vffi.cdef("const char *SSLeay_version(int type);")
    version_string = vffi.string(vffi.dlopen(libcrypto_path).SSLeay_version(0)).decode('utf-8') #### fails here
except (AttributeError):
    vffi = FFI()
    vffi.cdef("const char *OpenSSL_version(int type);")
    version_string = vffi.string(vffi.dlopen(libcrypto_path).OpenSSL_version(0)).decode('utf-8')

I suspect we are just picking a version that makes Catalina extremely upset, and so if we can pick a pinned version that would be better.

I posted something similar here: https://github.com/snowflakedb/snowflake-connector-python/issues/235

[wbond]

Can you provide a way to reproduce this segfault? Running the test suite doesn't seem to cause theses issues, so maybe we need to add something.

[worldwise001]

So from oscrypto import asymmetric was triggering it in the snowflake library. The code snippet from snowflake-connector-python that's relevant was in oscp_asn1crypto in verify_signature:

    def verify_signature(self, signature_algorithm, signature, cert, data):
        pubkey = asymmetric.load_public_key(cert.public_key).unwrap().dump() ### failing line
        rsakey = RSA.importKey(pubkey)
        signer = PKCS1_v1_5.new(rsakey)
        if signature_algorithm in SnowflakeOCSPAsn1Crypto.SIGNATURE_ALGORITHM_TO_DIGEST_CLASS:
            digest = \
                SnowflakeOCSPAsn1Crypto.SIGNATURE_ALGORITHM_TO_DIGEST_CLASS[
                    signature_algorithm].new()

[worldwise001]

I created a PR with a workaround for them too: https://github.com/snowflakedb/snowflake-connector-python/pull/241

[worldwise001]

I have been doing some more digging (to see if I can make a fix PR) and found the following w.r.t. libcrypto libraries on catalina:

Wed 20-11 (shh@shh-at-square) : /usr/lib ls -la libcrypto*
-rwxr-xr-x  1 root  wheel  1095424 Oct 23 17:40 libcrypto.0.9.7.dylib
-rwxr-xr-x  1 root  wheel  1390720 Oct 23 17:40 libcrypto.0.9.8.dylib
-rwxr-xr-x  1 root  wheel  1488080 Oct 23 17:40 libcrypto.35.dylib
-rwxr-xr-x  1 root  wheel  1500560 Oct 23 17:40 libcrypto.41.dylib
-rwxr-xr-x  1 root  wheel  1500592 Oct 23 17:40 libcrypto.42.dylib
-rwxr-xr-x  1 root  wheel  1519664 Oct 23 17:40 libcrypto.44.dylib
-rwxr-xr-x  1 root  wheel    32992 Oct 23 17:40 libcrypto.dylib

Based on the filesize, I suspect libcrypto.dylib unversioned is super small and its only purpose is to raise SIGABRT with a message telling people to use versioned libraries instead.

[worldwise001]

I also just filed https://bugs.python.org/issue38873 with python ctypes. I'm not sure if it's something they should fix, but I wonder if they can do us a favor and provide a means to specify version or something.

[wbond]

I wonder what is different about my machine that I don't get the abort.

Either way, we can just update it to use the versioned libcrypto as far as I am concerned. You don't happen to have a 10.14 install around do you? I can ensure we use a versioned lib on 10.15, but it wouldn't be terrible to include older releases if they have stable versions also.

[worldwise001]

I asked a coworker who is still on 10.14 to list his /usr/lib/libcrypto:

/usr/lib/libcrypto.0.9.7.dylib
/usr/lib/libcrypto.0.9.8.dylib
/usr/lib/libcrypto.35.dylib
/usr/lib/libcrypto.41.dylib
/usr/lib/libcrypto.42.dylib
/usr/lib/libcrypto.dylib

So 44 appears to only be on catalina at least/

[wbond]

Based on https://bugs.python.org/issue38814, I tend to doubt python core will deal with the issue, which I don't mind.

[worldwise001]

Opened this for you: https://github.com/wbond/oscrypto/pull/36

[wbond]

The only other useful thing would be to know what version of libressl the various versioned libcryptos are. Not that I expect any of them to break the APIs being used, since they are fairly pedestrian.

[worldwise001]

Wed 20-11 (shh@shh-at-square) : /usr/lib for i in `ls libcrypto.*.dylib`; do echo -n $i:; strings $i | grep libressl | head -n1 | cut -d'/' -f9; echo; done
libcrypto.0.9.7.dylib:
libcrypto.0.9.8.dylib:
libcrypto.35.dylib:libressl-2.2

libcrypto.41.dylib:libressl-2.5

libcrypto.42.dylib:libressl-2.6

libcrypto.44.dylib:libressl-2.8

[wbond]

Awesome - thanks for digging that up!

[worldwise001]

Any chance we can cut a release with this fix? It’s blocking my development right now. Keep me posted and thank you for your extremely prompt response!

[wbond]

I just need the CI on master to pass. I updated the CI config to pull in some revised matrix assignments across different CI providers that I made for asn1crypto. The major benefit being running the tests on 10.15 via GitHub Actions, however something is broken in regards to Circle and the macOS tests there. I'll need to make sure those look good before the next release.

[wbond]

This fix was included with the 1.1.1 release

[greenhouse]

Hi, simply updating to python 3.8 manually from python.org solved this issue for me.

unfortunately none of the above had worked for me : / I usually choose to be late in updating OS's (and Xcode, etc.), exactly due these types of issues.

i of course had to reinstall all my packages, but that was no big deal. not sure yet if anything else was effected by updating to python 3.8, so far non of my projects seem effected (i've been using python3.7 for well over a year)

hope this helps someone!