Building for MacOS M1 (Arm)

[ramvdixit]

I am having trouble building the Python Wrapper. I have followed everything that's described in the link: https://github.com/open-quantum-safe/liboqs-python and It stops with errors at 95% when I run the third command: cmake --build liboqs/build --parallel 8.

I am building on Apple MacBook Pro M1 Silicon and I think, arm64 is the issue. I see that its building for x86/64 but the target is arm64.

Here is the error extract from the point of failure:

[ 95%] Built target internal ld: warning: ignoring file /usr/local/opt/openssl@1.1/lib/libcrypto.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64 [ 95%] Linking C static library ../lib/liboqs-internal.a Undefined symbols for architecture arm64: "_EVP_CIPHER_CTX_free", referenced from: _AES128_free_schedule in aes_ossl.c.o _AES256_CTR_inc_stream_iv in aes_ossl.c.o "_EVP_CIPHER_CTX_new", referenced from: _AES128_ECB_load_schedule in aes_ossl.c.o _AES256_ECB_load_schedule in aes_ossl.c.o _AES256_CTR_inc_init in aes_ossl.c.o _AES256_CTR_inc_stream_iv in aes_ossl.c.o [ 95%] Built target oqs-internal "_EVP_CIPHER_CTX_set_padding", referenced from: _AES128_ECB_load_schedule in aes_ossl.c.o _AES256_ECB_load_schedule in aes_ossl.c.o "_EVP_DigestFinal_ex", referenced from: _SHA2_sha256_inc_finalize in sha2_ossl.c.o _SHA2_sha384_inc_finalize in sha2_ossl.c.o _SHA2_sha512_inc_finalize in sha2_ossl.c.o _do_hash in sha2_ossl.c.o "_EVP_DigestInit_ex", referenced from: _SHA2_sha256_inc_init in sha2_ossl.c.o _SHA2_sha384_inc_init in sha2_ossl.c.o _SHA2_sha512_inc_init in sha2_ossl.c.o _do_hash in sha2_ossl.c.o "_EVP_DigestUpdate", referenced from: _SHA2_sha256_inc_blocks in sha2_ossl.c.o _SHA2_sha256_inc_finalize in sha2_ossl.c.o _SHA2_sha384_inc_blocks in sha2_ossl.c.o _SHA2_sha384_inc_finalize in sha2_ossl.c.o _SHA2_sha512_inc_blocks in sha2_ossl.c.o _SHA2_sha512_inc_finalize in sha2_ossl.c.o _do_hash in sha2_ossl.c.o ... "_EVP_EncryptFinal_ex", referenced from: _AES128_ECB_enc_sch in aes_ossl.c.o _AES256_CTR_inc_stream_iv in aes_ossl.c.o "_EVP_EncryptInit_ex", referenced from: _AES128_ECB_load_schedule in aes_ossl.c.o _AES256_ECB_load_schedule in aes_ossl.c.o _AES256_CTR_inc_iv in aes_ossl.c.o _AES256_CTR_inc_ivu64 in aes_ossl.c.o _AES256_CTR_inc_stream_iv in aes_ossl.c.o "_EVP_EncryptUpdate", referenced from: _AES128_ECB_enc_sch in aes_ossl.c.o _AES256_CTR_inc_stream_iv in aes_ossl.c.o _AES256_CTR_inc_stream_blks in aes_ossl.c.o "_EVP_MD_CTX_copy_ex", referenced from: _SHA2_sha256_inc_ctx_clone in sha2_ossl.c.o _SHA2_sha384_inc_ctx_clone in sha2_ossl.c.o _SHA2_sha512_inc_ctx_clone in sha2_ossl.c.o "_EVP_MD_CTX_free", referenced from: _SHA2_sha256_inc_finalize in sha2_ossl.c.o _SHA2_sha256_inc_ctx_release in sha2_ossl.c.o _SHA2_sha384_inc_finalize in sha2_ossl.c.o _SHA2_sha384_inc_ctx_release in sha2_ossl.c.o _SHA2_sha512_inc_finalize in sha2_ossl.c.o _SHA2_sha512_inc_ctx_release in sha2_ossl.c.o _do_hash in sha2_ossl.c.o ... "_EVP_MD_CTX_new", referenced from: _SHA2_sha256_inc_init in sha2_ossl.c.o _SHA2_sha384_inc_init in sha2_ossl.c.o _SHA2_sha512_inc_init in sha2_ossl.c.o _do_hash in sha2_ossl.c.o "_EVP_aes_128_ecb", referenced from: _oqs_aes_128_ecb in ossl_helpers.c.o "_EVP_aes_256_ctr", referenced from: _oqs_aes_256_ctr in ossl_helpers.c.o "_EVP_aes_256_ecb", referenced from: _oqs_aes_256_ecb in ossl_helpers.c.o "_EVP_sha256", referenced from: _oqs_sha256 in ossl_helpers.c.o "_EVP_sha384", referenced from: _oqs_sha384 in ossl_helpers.c.o "_EVP_sha3_256", referenced from: _oqs_sha3_256 in ossl_helpers.c.o "_EVP_sha3_384", referenced from: _oqs_sha3_384 in ossl_helpers.c.o "_EVP_sha3_512", referenced from: _oqs_sha3_512 in ossl_helpers.c.o "_EVP_sha512", referenced from: _oqs_sha512 in ossl_helpers.c.o "_EVP_shake128", referenced from: _oqs_shake128 in ossl_helpers.c.o "_EVP_shake256", referenced from: _oqs_shake256 in ossl_helpers.c.o "_RAND_bytes", referenced from: _OQS_randombytes_openssl in rand.c.o "_RAND_poll", referenced from: _OQS_randombytes_openssl in rand.c.o "_RAND_status", referenced from: _OQS_randombytes_openssl in rand.c.o ld: symbol(s) not found for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: [lib/liboqs.0.10.1-dev.dylib] Error 1 make[1]: [src/CMakeFiles/oqs.dir/all] Error 2 make: *** [all] Error 2

Is there something wrong with what I am doing? Please help!

Thanks, Ram

[cothan]

Hi,

I think it says OpenSSL is dependency is missing. Can you build liboqs alone in the instruction here: https://github.com/open-quantum-safe/liboqs?tab=readme-ov-file#linux-and-mac

Let me know if it works.

[vsoftco]

@ramvdixit can you remove liboqs and simply try pip install . from the root of liboqs-python? This should take care of dependencies and install/build liboqs for you automatically. I cannot reproduce the issue on my end (running macOS as well on arm64).

[skilfoy]

I'm also having this issue. I'm running macOS with an M1 chip. I tried pip install . as instructed, and it appears to install normally. Then, when I import oqs or run python3 liboqs-python/examples/kem.py in terminal, I run into this problem:

liboqs not found, installing it in /Users/seankilfoy/_oqs
5 4 3 2 1
{snip}

And then goes through a long install process only to error out at the end with the following error along with the last few lines prior to the error:

[ 99%] Built target sphincs_shake_256f_simple_avx2
[ 99%] Building C object src/CMakeFiles/oqs.dir/sig/sig.c.o
[100%] Building C object src/CMakeFiles/oqs.dir/kem/kem.c.o
[100%] Linking C shared library ../lib/liboqs.dylib
[100%] Built target internal
[100%] Linking C static library ../lib/liboqs-internal.a
ld: warning: ignoring file '/opt/homebrew/Cellar/openssl@3/3.3.1/lib/libcrypto.3.dylib': found architecture 'arm64', required architecture 'x86_64'
ld: Undefined symbols:
  _CRYPTO_THREAD_run_once, referenced from:
      _oqs_sha256 in ossl_helpers.c.o
      _oqs_sha384 in ossl_helpers.c.o
      _oqs_sha512 in ossl_helpers.c.o
      _oqs_shake128 in ossl_helpers.c.o
      _oqs_shake256 in ossl_helpers.c.o
      _oqs_sha3_256 in ossl_helpers.c.o
      _oqs_sha3_384 in ossl_helpers.c.o
      ...
  _EVP_CIPHER_fetch, referenced from:
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
  _EVP_CIPHER_free, referenced from:
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
  _EVP_DigestFinal_ex, referenced from:
      _do_hash in sha2_ossl.c.o
      _do_hash in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_finalize in sha2_ossl.c.o
      ...
  _EVP_DigestInit_ex, referenced from:
      _do_hash in sha2_ossl.c.o
      _do_hash in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_init in sha2_ossl.c.o
      ...
  _EVP_DigestUpdate, referenced from:
      _do_hash in sha2_ossl.c.o
      _do_hash in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_blocks in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_blocks in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_blocks in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_blocks in sha2_ossl.c.o
      ...
  _EVP_MD_CTX_copy_ex, referenced from:
      _OQS_SHA2_sha256_inc_ctx_clone in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_ctx_clone in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_ctx_clone in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_ctx_clone in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_ctx_clone in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_ctx_clone in sha2_ossl.c.o
  _EVP_MD_CTX_free, referenced from:
      _do_hash in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_ctx_release in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_ctx_release in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_finalize in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_ctx_release in sha2_ossl.c.o
      ...
  _EVP_MD_CTX_new, referenced from:
      _do_hash in sha2_ossl.c.o
      _OQS_SHA2_sha256_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha384_inc_init in sha2_ossl.c.o
      _OQS_SHA2_sha512_inc_init in sha2_ossl.c.o
  _EVP_MD_fetch, referenced from:
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      _oqs_fetch_ossl_objects in ossl_helpers.c.o
      ...
  _EVP_MD_free, referenced from:
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      _oqs_free_ossl_objects in ossl_helpers.c.o
      ...
  _RAND_bytes, referenced from:
      _OQS_randombytes_openssl in rand.c.o
  _RAND_poll, referenced from:
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
  _RAND_status, referenced from:
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
      _OQS_randombytes_openssl in rand.c.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/liboqs.0.10.0.dylib] Error 1
make[1]: *** [src/CMakeFiles/oqs.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Built target oqs-internal
make: *** [all] Error 2
Done installing liboqs
Could not load liboqs shared library

[vsoftco]

@skilfoy thanks, I'll take a look

[dstebila]

I notice that some of the logs reference the directory /usr/local/opt. I thought that, on Apple Silicon, brew is installed in /opt/homebrew. So perhaps there's an old install of x86 brew that got copied over from a backup restoration, or something like that?

[vsoftco]

tagging @ryjones

[vsoftco]

Also, make sure /opt/homebrew is first in the PATH, e.g., I have this line in my .profile

# Homebrew
export PATH=/opt/homebrew/bin:/opt/homebrew/sbin:$PATH
export CPATH=/opt/homebrew/include
export LIBRARY_PATH=/opt/homebrew/lib

In case you use clang++ from Homebrew (not the default Apple clang, which I recommend against), also add

export PATH=/opt/homebrew/opt/llvm/bin:$HOME/bin:$PATH
export CXX=clang++
export CC=clang

[ryjones]

On a completely clean MacOS VM, I did:

git clone --depth=1 https://github.com/open-quantum-safe/liboqs
cmake -S liboqs -B liboqs/build -DBUILD_SHARED_LIBS=ON
cmake --build liboqs/build --parallel 8
sudo cmake --build liboqs/build --target install

And it worked fine. All I had installed was XCode command line tools cmake from Brew.

% which brew 
/opt/homebrew/bin/brew
% which cmake
/opt/homebrew/bin/cmake

[planetf1]

I also tried on my less clean dev environment, albeit with setting CMAKE_INSTALL_PREFIX to avoid adding libraries to system directories, and with a corresponding update to the loader path. M1 mac. Homebrew (arm). resolved compiler is Apple clang 15

[vsoftco]

@ryjones probably one can remove the grep from your 4th line

[planetf1]

Related to @dstebila 's comment above - in my environment I have openSSL libraries under both /usr/local (x86_64) as well as /opt/homebrew (arm64)

What is the output of

cmake -S liboqs -B liboqs/build -DBUILD_SHARED_LIBS=ON

Mine is:

Found OpenSSL: /opt/homebrew/Cellar/openssl@3/3.3.1/lib/libcrypto.dylib (found suitable version "3.3.1", minimum required is "1.1.1")

If you are resolving to old or intel versions, you may need to install openssl in homebrew (arm) ?

[planetf1]

I see also @skilfoy 's comment about the auto-build of liboqs failing when python is used directly, and the error:

ld: warning: ignoring file '/opt/homebrew/Cellar/openssl@3/3.3.1/lib/libcrypto.3.dylib': found architecture 'arm64', required architecture 'x86_64'

Yet this is running on a arm mac, with what seems to be arm implementations of openssl , but implies the rest of the build was intel. Again the beginning of this log (where cmake does the configuration) could be useful to see in terms of what compiler and libraries are being picked up