search

signalapp / libsignal

Home to the Signal Protocol as well as other cryptographic primitives which make Signal possible.
GNU Affero General Public License v3.0
3.06k stars 362 forks source link

cross compile for armv7-unknown-linux-gnueabihf and i686-unknown-linux-gnu fails #481

Closed morph027 closed 1 year ago

morph027 commented 1 year ago

I was trying to fix the build pipeline for https://github.com/exquo/signal-libs-build/ (https://github.com/exquo/signal-libs-build/issues/9), where the targets armv7-unknown-linux-gnueabihf and i686-unknown-linux-gnu do not not compile. Looks like support has been dropped, or?

I've tried to build locally and it fails for boringssl (tries to compile for linux-x86_64 from what i see)

error: failed to run custom build command for `boring-sys v2.0.0 (https://github.com/signalapp/boring?branch=libsignal#e1c719c0)`

Caused by:
  process didn't exit successfully: `/tmp/libsignal-client/target/release/build/boring-sys-4a360c7578ec69ce/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=BORING_BSSL_PATH
  CMAKE_TOOLCHAIN_FILE_armv7-unknown-linux-gnueabihf = None
  CMAKE_TOOLCHAIN_FILE_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_armv7-unknown-linux-gnueabihf = None
  CMAKE_GENERATOR_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_armv7-unknown-linux-gnueabihf = None
  CMAKE_PREFIX_PATH_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_armv7-unknown-linux-gnueabihf = None
  CMAKE_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE = None
  CMAKE = None
  running: "cmake" "/root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl" "-DCMAKE_INSTALL_PREFIX=/tmp/libsignal-client/target/armv7-unknown-linux-gnueabihf/release/build/boring-sys-7b57a0b7fce7eeb5/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_ASM_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc" "-DCMAKE_BUILD_TYPE=Release"
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/libsignal-client/target/armv7-unknown-linux-gnueabihf/release/build/boring-sys-7b57a0b7fce7eeb5/out/build
  running: "cmake" "--build" "." "--target" "crypto" "--config" "Release" "--parallel" "12"
  Scanning dependencies of target crypto
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S.o
  [  3%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S.o
  [  3%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/md5-x86_64.S.o
  [  4%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S.o
  [  4%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S.o
  [  4%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S.o
  [  4%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S.o

  --- stderr
  warning: no toolchain file configured by boring-sys for armv7-unknown-linux-gnueabihf
  gmake: warning: -j12 forced in submake: resetting jobserver mode.
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S:427: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S:63: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S:852: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/md5-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/md5-x86_64.S:702: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S: gmake[3]: *** [CMakeFiles/crypto.dir/build.make:140: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S.o] Error 1
  Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S: /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S:343: Error: junk at end of line, first unrecognized character is `,'
  Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S:1127: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** Waiting for unfinished jobs....
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1633: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:205: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S.o] Error 1
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:114: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S.o] Error 1
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:166: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/md5-x86_64.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S:2506: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:153: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S.o] Error 1
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:192: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S.o] Error 1
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:75: CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S:3079: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:127: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S.o] Error 1
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:88: CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S:1749: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:218: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S:4543: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:179: CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/e1c719c/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S:8922: Error: junk at end of line, first unrecognized character is `,'
  gmake[3]: *** [CMakeFiles/crypto.dir/build.make:101: CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S.o] Error 1
  gmake[2]: *** [CMakeFiles/Makefile2:87: CMakeFiles/crypto.dir/all] Error 2
  gmake[1]: *** [CMakeFiles/Makefile2:94: CMakeFiles/crypto.dir/rule] Error 2
  gmake: *** [Makefile:124: crypto] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 2

  build script failed, must exit now', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
jrose-signal commented 1 year ago

:-/ Yeah, boring-sys does not have good arbitrary-cross-compilation support. We test x86_64 and aarch64 builds for Ubuntu, macOS, Windows, iOS, and Android, as well as armv7 + i686 Android, but everything else is extra. Patches welcome, though, either to the upstream boring-sys or to our fork.

(I believe i686-unknown-linux-gnu should actually work if you set up the cross-compilation environment variables per #482 or install gcc-multilib and g++-multilib. But armv7-Linux wasn't ever tested; none of Signal's official clients run there. Since armv7-Android works, though, it shouldn't be too much extra effort.)

morph027 commented 1 year ago

Will probably have a look, just for curiosity. Personally, i'm not interested in arm7-linux builds ;) Was just trying to help fix @exquo github actions.

bbernhard commented 1 year ago

I am currently running into the same issue when cross compiling on amd64 for armv7. On a first glance it looks like this issue could be cross compilation related.

I am wondering if it would work when libsignal-client is directly compiled on a Raspberry Pi with a 32bit OS. It would probably take a while to compile (hopefully the Pi has enough memory) and it wouldn't be as convenient as the cross compilation, but at least there would be a way to build it for armv7.

Has maybe someone already tried that?

MichaelBitard commented 1 year ago

thanks to @tholland15, the libsignal can be built on those architecture on this project: https://github.com/agileek/hassio-addons

He/she created a github action workflows that builds libsignal on target virtual machines:

https://github.com/agileek/hassio-addons/pull/47 and https://github.com/agileek/hassio-addons/pull/49

exquo commented 1 year ago

Following the advice in https://github.com/signalapp/libsignal/issues/482#issuecomment-1220896664 (thanks!), it builds successfully for the aarch64-linux-gnu and i686-linux-gnu targets. But the armv7-linux-gnueabihf still fails..

I have tried to troubleshoot this by compiling the boring lib itself, and it builds fine for the armv7-Linux target (example run: https://github.com/exquo/signal-libs-build/actions/runs/3411698940/jobs/5676237923). But the build of libsignal in the very same environment fails on the building boring-sys step (https://github.com/exquo/signal-libs-build/actions/runs/3411284904/jobs/5675310584).

The build command used:

env $BUILD_ENV_VARS cargo build --release --verbose  $CARGO_FLAGS
env:
    CARGO_BUILD_TARGET: armv7-unknown-linux-gnueabihf
    RUSTFLAGS: -C link-arg=-s   
    CARGO_FLAGS: -p libsignal-jni
    BUILD_ENV_VARS:  CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ CPATH=/usr/arm-linux-gnueabihf/include

The cross-compilation tools are installed:

sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf

The build fails with

2022-11-07T14:45:03.7607224Z error: failed to run custom build command for `boring-sys v2.0.0 (https://github.com/signalapp/boring?branch=libsignal#b95cb545)`
<...>
2022-11-07T14:45:03.7630948Z   warning: no toolchain file configured by boring-sys for armv7-unknown-linux-gnueabihf
Full trace ``` error: failed to run custom build command for `boring-sys v2.0.0 (https://github.com/signalapp/boring?branch=libsignal#b95cb545)` Caused by: process didn't exit successfully: `/home/runner/work/signal-libs-build/signal-libs-build/target/release/build/boring-sys-bec5709c06bf1b54/build-script-build` (exit status: 101) --- stdout cargo:rerun-if-env-changed=BORING_BSSL_PATH CMAKE_TOOLCHAIN_FILE_armv7-unknown-linux-gnueabihf = None CMAKE_TOOLCHAIN_FILE_armv7_unknown_linux_gnueabihf = None TARGET_CMAKE_TOOLCHAIN_FILE = None CMAKE_TOOLCHAIN_FILE = None CMAKE_GENERATOR_armv7-unknown-linux-gnueabihf = None CMAKE_GENERATOR_armv7_unknown_linux_gnueabihf = None TARGET_CMAKE_GENERATOR = None CMAKE_GENERATOR = None CMAKE_PREFIX_PATH_armv7-unknown-linux-gnueabihf = None CMAKE_PREFIX_PATH_armv7_unknown_linux_gnueabihf = None TARGET_CMAKE_PREFIX_PATH = None CMAKE_PREFIX_PATH = None CMAKE_armv7-unknown-linux-gnueabihf = None CMAKE_armv7_unknown_linux_gnueabihf = None TARGET_CMAKE = None CMAKE = None running: "cmake" "/home/runner/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl" "-DCMAKE_INSTALL_PREFIX=/home/runner/work/signal-libs-build/signal-libs-build/target/armv7-unknown-linux-gnueabihf/release/build/boring-sys-23b3e4b548f228ed/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_ASM_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc" "-DCMAKE_BUILD_TYPE=Release" -- The C compiler identification is GNU 7.5.0 -- The CXX compiler identification is GNU 7.5.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/arm-linux-gnueabihf-gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is GNU -- Found assembler: /usr/bin/arm-linux-gnueabihf-gcc -- Configuring done -- Generating done -- Build files have been written to: /home/runner/work/signal-libs-build/signal-libs-build/target/armv7-unknown-linux-gnueabihf/release/build/boring-sys-23b3e4b548f228ed/out/build running: "cmake" "--build" "." "--target" "crypto" "--config" "Release" "--parallel" "2" Scanning dependencies of target crypto [ 0%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o [ 1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o CMakeFiles/crypto.dir/build.make:87: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o' failed CMakeFiles/crypto.dir/build.make:74: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o' failed CMakeFiles/Makefile2:86: recipe for target 'CMakeFiles/crypto.dir/all' failed CMakeFiles/Makefile2:93: recipe for target 'CMakeFiles/crypto.dir/rule' failed Makefile:124: recipe for target 'crypto' failed --- stderr warning: no toolchain file configured by boring-sys for armv7-unknown-linux-gnueabihf make: warning: -jN forced in submake: disabling jobserver mode. /home/runner/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S: Assembler messages: /home/runner/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S:3079: Error: junk at end of line, first unrecognized character is `,' make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o] Error 1 make[3]: *** Waiting for unfinished jobs.... /home/runner/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S: Assembler messages: /home/runner/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1633: Error: junk at end of line, first unrecognized character is `,' make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o] Error 1 make[2]: *** [CMakeFiles/crypto.dir/all] Error 2 make[1]: *** [CMakeFiles/crypto.dir/rule] Error 2 make: *** [crypto] Error 2 thread 'main' panicked at ' command did not execute successfully, got: exit status: 2 build script failed, must exit now', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace warning: build failed, waiting for other jobs to finish... ##[error]Process completed with exit code 101. ``` (Running with `RUST_BACKTRACE=1` did not give any further hints, AFAICT)

Any insights into why it fails to compile boring as a part of libsignal, while the standalone boring (the signalapp/boring fork, libsignal branch - same as in libsignal's own dependencies) builds successfully, would be appreciated!

bbernhard commented 1 year ago

I wasn't able to cross compile it either. It should work by compiling in an emulated armv7 environment (with QEMU). Unfortunately, I wasn't able to set up an armv7 QEMU debian machine either (only spent a few hours on this, but I think it should be doable with enough time). In the end, I just used a Raspberry Pi and compiled it there

exquo commented 1 year ago

It should work by compiling in an emulated armv7 environment (with QEMU).

Thanks! But I would like to avoid emulation if possible. Rust on its own should have very natural cross-compilation capabilities.

morph027 commented 1 year ago

Yes, this works (DOCKER_DEFAULT_PLATFORM=linux/arm/v7 docker run --rm -it arm32v7/ubuntu:focal), but sure, spinning up an QEMU w/ arm7 feels like overkill if it should work without.

As @jrose-signal already pointed out, there's no support yet (see https://github.com/signalapp/boring/blob/libsignal/boring-sys/build.rs#L254-L275). Will try to adapt for armv7 and see if it works.

jrose-signal commented 1 year ago

Yeah, I'm not sure offhand what's different between the boring build and the libsignal build here; I would expect them both to fail. I would say it's possible your cache is getting in the way, since I had problems with incremental builds when changing CMake options in the boring repo. You could try not saving the target directory and see if that changes anything. But in both specific runs you've linked, the cache says it's not being used.

morph027 commented 1 year ago 
apt-get -qqy install build-essential cmake clang libprotobuf-dev protobuf-compiler git gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf
git clone -b v0.21.1 https://github.com/signalapp/libsignal-client /tmp/libsignal-client
cd /tmp/libsignal-client
sed -i "s#github.com/signalapp/boring', branch = 'libsignal'#github.com/morph027/boring', branch = 'libsignal-arm'#" Cargo.toml rust/attest/Cargo.toml
export CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_LINKER=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ CC=arm-linux-gnueabihf-gcc RUSTFLAGS="-C link-arg=-s" CARGO_BUILD_TARGET=armv7-unknown-linux-gnueabihf
cargo build --release --verbose -p libsignal-jni

Results in:

# file target/armv7-unknown-linux-gnueabihf/release/libsignal_jni.so 
target/armv7-unknown-linux-gnueabihf/release/libsignal_jni.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=566043b3ec9418db43b0f9ee41f92a19152ad170, stripped

PR => https://github.com/signalapp/boring/pull/15

jrose-signal commented 1 year ago

I've rebased and merged that PR; it'll be referenced in the next release of libsignal. Thank you for providing that, and testing it!

morph027 commented 1 year ago

Just tried to compile again and it fails (again :wink: ) ... Pinning signalapp/boring to https://github.com/signalapp/boring/commit/73ee9d70853fa51e20baece584e03c81d0dc0af8 makes it work. I do not really see a huge impact from https://github.com/signalapp/boring/commit/25e1dd8fa519ae90772008fbd4a284582096fece

Error

error: failed to run custom build command for `boring-sys v2.0.0 (https://github.com/signalapp/boring?branch=libsignal#b95cb545)`

Caused by:
  process didn't exit successfully: `/tmp/libsignal-client/target/release/build/boring-sys-bec5709c06bf1b54/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=BORING_BSSL_PATH
  CMAKE_TOOLCHAIN_FILE_armv7-unknown-linux-gnueabihf = None
  CMAKE_TOOLCHAIN_FILE_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_armv7-unknown-linux-gnueabihf = None
  CMAKE_GENERATOR_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_armv7-unknown-linux-gnueabihf = None
  CMAKE_PREFIX_PATH_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_armv7-unknown-linux-gnueabihf = None
  CMAKE_armv7_unknown_linux_gnueabihf = None
  TARGET_CMAKE = None
  CMAKE = None
  running: "cmake" "/root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl" "-DCMAKE_INSTALL_PREFIX=/tmp/libsignal-client/target/armv7-unknown-linux-gnueabihf/release/build/boring-sys-23b3e4b548f228ed/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -march=armv7-a -mfpu=vfpv3-d16" "-DCMAKE_ASM_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc" "-DCMAKE_BUILD_TYPE=Release"
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/libsignal-client/target/armv7-unknown-linux-gnueabihf/release/build/boring-sys-23b3e4b548f228ed/out/build
  running: "cmake" "--build" "." "--target" "crypto" "--config" "Release" "--parallel" "12"
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S.o
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o
  [  1%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S.o
  [  2%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S.o
  [  3%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S.o
  [  3%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/md5-x86_64.S.o
  [  4%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S.o
  [  4%] Building ASM object CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S.o
  CMakeFiles/crypto.dir/build.make:74: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:178: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S.o' failed
  CMakeFiles/crypto.dir/build.make:87: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:191: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S.o' failed
  CMakeFiles/crypto.dir/build.make:126: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:204: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:165: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/md5-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:152: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:217: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S.o' failed
  CMakeFiles/crypto.dir/build.make:113: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:139: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S.o' failed
  CMakeFiles/crypto.dir/build.make:100: recipe for target 'CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S.o' failed
  CMakeFiles/Makefile2:86: recipe for target 'CMakeFiles/crypto.dir/all' failed
  CMakeFiles/Makefile2:93: recipe for target 'CMakeFiles/crypto.dir/rule' failed
  Makefile:124: recipe for target 'crypto' failed

  --- stderr
  warning: no toolchain file configured by boring-sys for armv7-unknown-linux-gnueabihf
  make: warning: -jN forced in submake: disabling jobserver mode.
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/chacha/chacha-x86_64.S:1633: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S:4543: Error: junk at end of line, first unrecognized character is `,'
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/chacha/chacha-x86_64.S.o] Error 1
  make[3]: *** Waiting for unfinished jobs....
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256-x86_64-asm.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S:3079: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S:343: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S:2506: Error: junk at end of line, first unrecognized character is `,'
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/aes128gcmsiv-x86_64.S.o] Error 1
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/p256_beeu-x86_64-asm.S.o] Error 1
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-x86_64.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/md5-x86_64.S: /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S:63: Error: junk at end of line, first unrecognized character is `,'
  Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/md5-x86_64.S:702: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S:1127: Error: junk at end of line, first unrecognized character is `,'
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rdrand-x86_64.S.o] Error 1
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/md5-x86_64.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S:1749: Error: junk at end of line, first unrecognized character is `,'
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-x86_64.S.o] Error 1
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/rsaz-avx2.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S:852: Error: junk at end of line, first unrecognized character is `,'
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/aesni-gcm-x86_64.S.o] Error 1
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S:427: Error: junk at end of line, first unrecognized character is `,'
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S: Assembler messages:
  /root/.cargo/git/checkouts/boring-b37daebd62069023/b95cb54/boring-sys/deps/boringssl/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S:8922: Error: junk at end of line, first unrecognized character is `,'
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/fipsmodule/ghash-ssse3-x86_64.S.o] Error 1
  make[3]: *** [CMakeFiles/crypto.dir/linux-x86_64/crypto/cipher_extra/chacha20_poly1305_x86_64.S.o] Error 1
  make[2]: *** [CMakeFiles/crypto.dir/all] Error 2
  make[1]: *** [CMakeFiles/crypto.dir/rule] Error 2
  make: *** [crypto] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 2

  build script failed, must exit now', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
jrose-signal commented 1 year ago

Yeah, no new release yet, I'm sorry! Waiting on a completely unrelated internal change. (In general we try not to do releases that don't make it into any app builds, purely from a project management / overhead perspective for our small team. But it has been almost two months since you came up with the fix, so we'll probably do one soon anyway.)

morph027 commented 1 year ago

No hurry, i guess it's not related to this project but to boring fork, as the "fix" was in there.

https://github.com/signalapp/boring/commits/libsignal

libsignal does use this branch (see https://github.com/signalapp/libsignal/blob/main/Cargo.toml#L25).

Just tried a new build this week and it failed. There's only one commit on top of my/our fix.

jrose-signal commented 1 year ago

If you're building in the libsignal repo, you're using libsignal's Cargo.lock, so the branch doesn't include your fix. cargo update -p boring before building should advance to the latest version.

If you're not building in the libsignal repo, hm, I'm not sure what's wrong!

jrose-signal commented 1 year ago

Released in v0.22.0!

exquo commented 1 year ago

Thank you very much @morph027 for creating the fix, and @jrose-signal for including it in the new release! ARMv7 builds successfully now 🎉️