libcrypto on ARMv8 (Rpi4) fails with Illegal Instruction

ulfworsoe commented 3 years ago

Actual Behavior

On Ubuntu 19.10, with latest Miniconda (Miniconda3-py39_4.9.2-Linux-aarch64) on a Raspberry Pi 4, importing hashlibfails with "Illegal instruction (core dumped)".

EDIT: This only happens after having installed conda-build.

Expected Behavior

Hashlib should load.

Steps to Reproduce

EDIT:

Install conda-build (which updates openssl
Activate and Start python
import hashlib

Anaconda or Miniconda version:

Miniconda3-py39_4.9.2-Linux-aarch64

Operating System:

Ubuntu 19.10, running on Raspberry Pi 4

`conda info`

conda info produces same error, "illegal instruction". It is casued by an issue in libcrypto. This is what gdb reports:

Program received signal SIGILL, Illegal instruction.
0x0000fffff6aff6bc in getrn () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
(gdb) bt
#0  0x0000fffff6aff6bc in getrn () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#1  0x0000fffff6affa4c in OPENSSL_LH_insert () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#2  0x0000fffff6b0a820 in OBJ_NAME_add () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#3  0x0000fffff6af373c in EVP_add_digest () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#4  0x0000fffff6ae0e54 in openssl_add_all_digests_int () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#5  0x0000fffff6afcb3c in ossl_init_add_all_digests_ossl_ () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#6  0x0000fffff7fae0d0 in __pthread_once_slow (once_control=0xfffff6c66594 <add_all_digests>, init_routine=0xfffff6afcb28 <ossl_init_add_all_digests_ossl_>) at pthread_once.c:116
#7  0x0000fffff6b531b4 in CRYPTO_THREAD_run_once () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#8  0x0000fffff6afd23c in OPENSSL_init_crypto () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#9  0x0000fffff6af3a60 in EVP_MD_do_all () from /home/ulfw/miniconda3-aarch64/lib/python3.9/lib-dynload/../../libcrypto.so.1.1
#10 0x0000fffff6c6d1f0 in hashlib_md_meth_names (module=0xfffff6c9e220) at /usr/local/src/conda/python-3.9.1/Modules/_hashopenssl.c:1775
#11 PyInit__hashlib () at /usr/local/src/conda/python-3.9.1/Modules/_hashopenssl.c:2135

A quick search on the subject indicates that libcrypto installs a signal handler for SIGILL on startup, then probes the processors capabilities, which can produce a SIGILL. The signal GDB catches appears to not be handled by a handler and causes the executable to crash.

chenghlee commented 3 years ago

The linux-aarch64 packages in default are built to target the Neoverse N1/N2 microarchitectures; in particular, we use -march=armv8.2-a+fp16+rcpc+dotprod+crypto as one of our default compiler flags, and I wouldn't be surprised if that inserted instructions into libcrypto that the RPi's Cortex-A72 doesn't implement.

The diversity of ARM CPUs is a major challenge as we expand the conda package ecosystem to support it. Conda currently does not have a simple way to select packages based on specific CPU features (see conda/conda#9930), but that's seems like something we will need to properly support ARM platforms.

ulfworsoe commented 3 years ago

I expect it is probably something like that. The arm crypto extension isn't supported in Rpi4, so that might very well be the culprit.

ContinuumIO / anaconda-issues