[QUESTION] library link error when cross-compiling

[JacobLeeYootaek]

I'm trying to cross-compile the library for arm but it fails while Linking C executable test_srtp. How can I fix it?

The command

export CC=arm-linux-gnueabihf-gcc export CXX=arm-linux-gnueabihf-g++ export LD=arm-linux-gnueabihf-ld export AR=arm-linux-gnueabihf-ar

cmake .. -DBUILD_TEST=TRUE -DBUILD_STATIC_LIBS=TRUE -DBUILD_OPENSSL=TRUE -DBUILD_OPENSSL_PLATFORM=linux-generic32 -DOPENSSL_EXTRA="no-async;no-asm" -DBUILD_LIBSRTP_HOST_PLATFORM=x86_64-pc-linux-gnu -DBUILD_LIBSRTP_DESTINATION_PLATFORM=arm-linux-gnueabihf

The error

[ 66%] Performing build step for 'project_libsrtp' Scanning dependencies of target srtp2 [ 4%] Building C object CMakeFiles/srtp2.dir/srtp/ekt.c.o [ 8%] Building C object CMakeFiles/srtp2.dir/srtp/srtp.c.o [ 12%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/cipher.c.o [ 16%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/null_cipher.c.o [ 20%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/aes_icm_ossl.c.o [ 24%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/aes_gcm_ossl.c.o [ 28%] Building C object CMakeFiles/srtp2.dir/crypto/hash/auth.c.o [ 32%] Building C object CMakeFiles/srtp2.dir/crypto/hash/null_auth.c.o [ 36%] Building C object CMakeFiles/srtp2.dir/crypto/hash/hmac_ossl.c.o [ 40%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/alloc.c.o [ 44%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/crypto_kernel.c.o [ 48%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/err.c.o [ 52%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/key.c.o [ 56%] Building C object CMakeFiles/srtp2.dir/crypto/math/datatypes.c.o [ 60%] Building C object CMakeFiles/srtp2.dir/crypto/math/stat.c.o [ 64%] Building C object CMakeFiles/srtp2.dir/crypto/replay/rdb.c.o [ 68%] Building C object CMakeFiles/srtp2.dir/crypto/replay/rdbx.c.o [ 72%] Building C object CMakeFiles/srtp2.dir/crypto/replay/ut_sim.c.o [ 76%] Linking C static library libsrtp2.a [ 76%] Built target srtp2 Scanning dependencies of target test_srtp [ 80%] Building C object CMakeFiles/test_srtp.dir/test/test_srtp.c.o [ 84%] Linking C executable test_srtp /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_lock_new': threads_pthread.c:(.text+0x3a): undefined reference topthread_rwlock_init' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_read_lock': threads_pthread.c:(.text+0x5a): undefined reference topthread_rwlock_rdlock' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_write_lock': threads_pthread.c:(.text+0x6a): undefined reference topthread_rwlock_wrlock' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_unlock': threads_pthread.c:(.text+0x7a): undefined reference topthread_rwlock_unlock' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_lock_free': threads_pthread.c:(.text+0x8e): undefined reference topthread_rwlock_destroy' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_run_once': threads_pthread.c:(.text+0xaa): undefined reference topthread_once' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_init_local': threads_pthread.c:(.text+0xba): undefined reference topthread_key_create' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_set_local': threads_pthread.c:(.text+0xd4): undefined reference topthread_setspecific' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_cleanup_local': threads_pthread.c:(.text+0xe4): undefined reference topthread_key_delete' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function openssl_init_fork_handlers': threads_pthread.c:(.text+0x12a): undefined reference topthread_once' /home/jacob/Workspace/toolchain/arm-linux-gnueabihf/bin/ld: /home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/lib/libcrypto.a(threads_pthread.o): in function CRYPTO_THREAD_get_local': threads_pthread.c:(.text+0xca): undefined reference topthread_getspecific' collect2: error: ld returned 1 exit status CMakeFiles/test_srtp.dir/build.make:96: recipe for target 'test_srtp' failed CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/test_srtp.dir/all' failed make[5]: ** [test_srtp] Error 1 make[4]: [CMakeFiles/test_srtp.dir/all] Error 2 Makefile:140: recipe for target 'all' failed make[3]: [all] Error 2 CMakeFiles/project_libsrtp.dir/build.make:111: recipe for target 'build/src/project_libsrtp-stamp/project_libsrtp-build' failed make[2]: [build/src/project_libsrtp-stamp/project_libsrtp-build] Error 2 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/project_libsrtp.dir/all' failed Makefile:83: recipe for target 'all' failed make[1]: [CMakeFiles/project_libsrtp.dir/all] Error 2 make: [all] Error 2 CMake Error at CMake/Utilities.cmake:65 (message): CMake step for libsrtp failed: 2 Call Stack (most recent call first): CMakeLists.txt:129 (build_dependency) -- Configuring incomplete, errors occurred! See also "/home/jacob/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/build/CMakeFiles/CMakeOutput.log".**

[MushMal]

@JacobLeeYootaek seems like an issue with compiling libWebSockets.

Simply searched the net: https://github.com/hlandau/acme.t/issues/1

See if this fixes. Perhaps a good question to ask libWebSocket forums.

[JacobLeeYootaek]

@MushMal Sorry the issue about sys/capability.h was that I pasted wrong error at the first. My issue is about linking test_srtp.

[MushMal]

I am not familiar with test_srtp - what is this?

Looking at your log again, I see multiple errors with pthread, etc..

Try to follow similar steps as our Travis CI runs for cross-compiling:

https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/blob/master/.travis.yml#L165

[JacobLeeYootaek]

@MushMal I don't know what test_srtp is too. I asked the question because the compiling is stuck at there.

I followed ARM Cross-compilation of Travis CI and I want a static library so I tried with this command: cmake .. -DBUILD_STATIC_LIBS=TRUE -DBUILD_OPENSSL=TRUE -DBUILD_OPENSSL_PLATFORM=linux-generic32 -DBUILD_LIBSRTP_HOST_PLATFORM=x86_64-pc-linux-gnu -DBUILD_LIBSRTP_DESTINATION_PLATFORM=arm-linux-gnueabihf

But it is still the same.

[MushMal]

As I am unable to reproduce this I can’t give any precise instructions. Perhaps your tool chain doesn’t have the library definition for the pthread? I am also unfamiliar with the test_srtp.

How can we bisect the issue?

[JacobLeeYootaek]

I am using this toolchain: gcc-arm-8.2-2018.08-x86_64-arm-linux-gnueabihf from https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads/8-2-2018-08

Could you try it with this?

Maybe you should copy capability.h to /gcc-arm-8.2-2018.08-x86_64-arm-linux-gnueabihf/arm-linux-gnueabihf/libc/usr/include/sys/capability.h Because an error occurs with it. I referred this repository: https://github.com/mhiramat/libcap

[disa6302]

@JacobLeeYootaek ,

Make sure you set the appropriate toolchain for BUILD_LIBSRTP_DESTINATION_PLATFORM. The best way to debug this for your platform and toolchain is to first cross compile the libsrtp library on ARM and see if it generates errors. If that generates errors, it is most likely that KVS would not compile as well. We might not be able to give targeted support for all toolchains and platforms.

[JacobLeeYootaek]

@disa6302 I cross compiled only libsrtp library but it is succeeded. So i'm wondering why I failed on amazon-kinesis-video-streams-webrtc-sdk-c. Do you have any idea?

When I cross compile only the libsrtp library: Scanning dependencies of target srtp2 [ 3%] Building C object CMakeFiles/srtp2.dir/srtp/ekt.c.o [ 7%] Building C object CMakeFiles/srtp2.dir/srtp/srtp.c.o [ 11%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/cipher.c.o [ 15%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/null_cipher.c.o [ 19%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/aes.c.o [ 23%] Building C object CMakeFiles/srtp2.dir/crypto/cipher/aes_icm.c.o [ 26%] Building C object CMakeFiles/srtp2.dir/crypto/hash/auth.c.o [ 30%] Building C object CMakeFiles/srtp2.dir/crypto/hash/null_auth.c.o [ 34%] Building C object CMakeFiles/srtp2.dir/crypto/hash/hmac.c.o [ 38%] Building C object CMakeFiles/srtp2.dir/crypto/hash/sha1.c.o [ 42%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/alloc.c.o [ 46%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/crypto_kernel.c.o [ 50%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/err.c.o [ 53%] Building C object CMakeFiles/srtp2.dir/crypto/kernel/key.c.o [ 57%] Building C object CMakeFiles/srtp2.dir/crypto/math/datatypes.c.o [ 61%] Building C object CMakeFiles/srtp2.dir/crypto/math/stat.c.o [ 65%] Building C object CMakeFiles/srtp2.dir/crypto/replay/rdb.c.o [ 69%] Building C object CMakeFiles/srtp2.dir/crypto/replay/rdbx.c.o [ 73%] Building C object CMakeFiles/srtp2.dir/crypto/replay/ut_sim.c.o [ 76%] Linking C static library libsrtp2.a [ 76%] Built target srtp2 Scanning dependencies of target test_srtp [ 80%] Building C object CMakeFiles/test_srtp.dir/test/test_srtp.c.o [ 84%] Linking C executable test_srtp [ 84%] Built target test_srtp Scanning dependencies of target srtp_driver [ 88%] Building C object CMakeFiles/srtp_driver.dir/test/srtp_driver.c.o [ 92%] Building C object CMakeFiles/srtp_driver.dir/test/util.c.o [ 96%] Building C object CMakeFiles/srtp_driver.dir/test/getopt_s.c.o [100%] Linking C executable srtp_driver [100%] Built target srtp_driver jacob@ubuntu:~/Workspace/Github/amazon-kinesis-video-streams-webrtc-sdk-c/open-source/libsrtp/build/src/project_libsrtp-build$ file test_srtp test_srtp: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=571829eb4ace61abefc1f327b338837f4049efb7, for GNU/Linux 3.2.0, with debug_info, not stripped

[MushMal]

Hard to tell. Perhaps some path issues or some configuration with that tool chain. Try different options and search the web to see why for example pthread lib is missing.

[MushMal]

BTW, can you try this one out? https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/pull/1048/files

Seems that libsrtp has an issue with gcc-10 so had to patch it

[JacobLeeYootaek]

I pulled the new commit and retried it but it is the same.

[MushMal]

Ok, can you check if you are failing to build srtp library or you are having an issue with KVS itself? Try to start from an absolutely clean repository.

If the issue is with srtp library then perhaps you can cut an issue against the srtp library repository to get more help

[JacobLeeYootaek]

I just solved it by using mbedtls instead of OpenSSL and it works well. The issue seems like that ${LIBS} has wrong option but I couldn't find where I should fix.

Anyway, thank you for helping me :D