search

rust-lang / libz-sys

Rust crate package to link to a system libz (zlib)
Apache License 2.0
119 stars 77 forks source link

Failed to compile libz-ng-sys v1.1.8 on armv7-unknown-linux-gnueabihf #108

Open NobodyXu opened 2 years ago

NobodyXu commented 2 years ago

Running cmd

cross build --target armv7-unknown-linux-gnueabihf --profile release --no-default-features --features zlib-ng,static,rustls,trust-dns,fancy-no-backtrace,log_release_max_level_debug -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort

on cargo-binstall v0.15.0 failed with error:

  /usr/lib/gcc-cross/arm-linux-gnueabihf/5/include/arm_neon.h:31:2: error: #error You must enable NEON instructions (e.g. -mfloat-abi=softfp -mfpu=neon) to use arm_neon.h
   #error You must enable NEON instructions (e.g. -mfloat-abi=softfp -mfpu=neon) to use arm_neon.h

From CI failure.

and

  In file included from /cargo/registry/src/index.crates.io-e139d0d48fed7772/libz-ng-sys-1.1.8/src/zlib-ng/compare256.c:7:0:
  /cargo/registry/src/index.crates.io-e139d0d48fed7772/libz-ng-sys-1.1.8/src/zlib-ng/fallback_builtins.h:102:15: error: unknown type name 'uint16x8x4_t'
   static inline uint16x8x4_t vld1q_u16_x4(uint16_t *a) {

According to this comment from @Alexhuszagh:

This isn't a cross bug: it's an issue with the library you're trying to use: ARMv7HF defaults to using VFPv3, not NEON, and refuses to compile NEON intrinsics unless you basically force it to. You'd have to add -C target-feature=+neon to RUSTFLAGS, and even then I doubt it would work. If you want an ARMv7 hard-float target that supports NEON by default, use thumbv7neon-unknown-linux-gnueabihf. FYI: your library is also asking you to use -mfloat-abi=softfp which is definitely wrong here: it would have to be -mfloat-abi=hard.

There's something wrong with the libraries you are using, not with the toolchain itself.

libz-ng should not use NEON intrinsics on armv7hf.

Alexhuszagh commented 2 years ago

Just a clarification: it should be already using the hard-float ABI, but the error diagnostic asking you to use -mfloat-abi=softfp is just a generic suggestion and is misleading here. -C target-feature=+neon would be required for it to work, but I'm unsure if it would out-of-the-box (although it might, I'm just guessing here).

Alexhuszagh commented 2 years ago

FYI: this seems to be fixed with a more recent toolchain (GCC 9.4.0 and glibc 2.31) and fails with an older one (GCC 5.4.0 and glibc 2.23). I'm not sure if libz-sys supports older glibc versions?

thomcc commented 2 years ago

This caused issues with cmake-rs's CI in https://github.com/rust-lang/cmake-rs/pull/164.

Should cmake-rs be doing something to help here? It seems semi-reasonable that we could translate a subset of CARGO_CFG_TARGET_FEATURE into flags for C compilers (perhaps that would be better in cc though...)

Alexhuszagh commented 2 years ago

The feature detection for a hard-float ARMv7 target seems to be fixed with a more recent GCC version, so this should hopefully be fixed with the 0.3.0 release. I'll check out the PowerPC64 issue as well, and I'll try to triage roughly loosely what toolchain versions replicate the issue. Maybe this can help either cmake-rs or ccs-rs, since this isn't exclusively a cross issue and it would be nice to ensure any fixes are significantly comprehensive, if possible.

tackn77 commented 1 year ago

Hello.

I am building using armv8/rust to study deno on raspberry pi.

Since deno v1.36.1 my builds are failing. When I checked the build in the container, it was failing in cmake.

I was hoping that it would go away after a while, but there seems to be no improvement even after 3 months.

I will share it

# cargo -V
info: syncing channel updates for '1.73.0-aarch64-unknown-linux-gnu'
info: latest update on 2023-10-05, rust version 1.73.0 (cc66ad468 2023-10-03)
info: downloading component 'clippy'
info: downloading component 'rustfmt'
info: installing component 'clippy'
info: installing component 'rustfmt'
cargo 1.73.0 (9c4383fb5 2023-08-26)
error: failed to run custom build command for `libz-ng-sys v1.1.12`

Caused by:
  process didn't exit successfully: `/usr/local/src/deno/target/debug/build/libz-ng-sys-c39de38f793215a3/build-script-build_zng` (exit status: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_aarch64-unknown-linux-gnu = None
  CMAKE_TOOLCHAIN_FILE_aarch64_unknown_linux_gnu = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_aarch64-unknown-linux-gnu = None
  CMAKE_GENERATOR_aarch64_unknown_linux_gnu = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_aarch64-unknown-linux-gnu = None
  CMAKE_PREFIX_PATH_aarch64_unknown_linux_gnu = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_aarch64-unknown-linux-gnu = None
  CMAKE_aarch64_unknown_linux_gnu = None
  HOST_CMAKE = None
  CMAKE = None
  running: cd "/usr/local/src/deno/target/debug/build/libz-ng-sys-d165f15752aa611b/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libz-ng-sys-1.1.12/src/zlib-ng" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=OFF" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_INSTALL_PREFIX=/usr/local/src/deno/target/debug/build/libz-ng-sys-d165f15752aa611b/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Debug"

  --- stderr
  thread 'main' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  failed to execute command:
Byron commented 1 year ago

cmake seems to not produce an actual reason for its failure, and I wonder if there is more output if you'd execute command past running: … yourself? Maybe that allows to pass additional arguments to provide more information as well.

Besides that, another step could be to open a PR which adjust CI so that it runs into the same failure. From there I'd hope something can be done to fix it, either here or by contributing something to the cmake crate.

Thanks for your help.

tackn77 commented 1 year ago

Thanks for the response.

The only work to be done in docker build as follows is to execute the following commands. cargo build --release

I would like to learn more so that I can report more details.

Dockerfile

FROM arm64v8/rust AS build-rust
WORKDIR /usr/local/src

ARG DENO_VERSION=main
RUN git clone --recurse-submodules https://github.com/denoland/deno.git -b ${DENO_VERSION} --depth 1
RUN cd deno && CARGO_HOME=/cargo cargo check
RUN cd deno && CARGO_HOME=/cargo cargo build --release
tackn77 commented 1 year ago

I can't read the log. I tried to do a manual cargo check on the docker machine.

Add environment variables

export RUST_BACKTRACE=1
export CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true
export RUST_BACKTRACE=full

execution cargo check

root@b643aef843f5:/usr/local/src/deno# cargo check
   Compiling libz-ng-sys v1.1.12
   Compiling smartstring v1.0.1
   Compiling proc-macro-hack v0.5.20+deprecated
    Checking cpufeatures v0.2.9
error: failed to run custom build command for `libz-ng-sys v1.1.12`

Caused by:
  process didn't exit successfully: `/usr/local/src/deno/target/debug/build/libz-ng-sys-baa5f1ec6e401685/build-script-build_zng` (exit status: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_aarch64-unknown-linux-gnu = None
  CMAKE_TOOLCHAIN_FILE_aarch64_unknown_linux_gnu = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_aarch64-unknown-linux-gnu = None
  CMAKE_GENERATOR_aarch64_unknown_linux_gnu = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_aarch64-unknown-linux-gnu = None
  CMAKE_PREFIX_PATH_aarch64_unknown_linux_gnu = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_aarch64-unknown-linux-gnu = None
  CMAKE_aarch64_unknown_linux_gnu = None
  HOST_CMAKE = None
  CMAKE = None
  running: cd "/usr/local/src/deno/target/debug/build/libz-ng-sys-740be895971aef52/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/cargo/registry/src/index.crates.io-6f17d22bba15001f/libz-ng-sys-1.1.12/src/zlib-ng" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=OFF" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_INSTALL_PREFIX=/usr/local/src/deno/target/debug/build/libz-ng-sys-740be895971aef52/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Debug"

  --- stderr
  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  failed to execute command: No such file or directory (os error 2)
  is `cmake` not installed?

  build script failed, must exit now
  stack backtrace:
     0:     0xaaaaaf4a5d98 - std::backtrace_rs::backtrace::libunwind::trace::heaab0e590535aeb3
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
     1:     0xaaaaaf4a5d98 - std::backtrace_rs::backtrace::trace_unsynchronized::h89cc7ae9ebb707d7
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
     2:     0xaaaaaf4a5d98 - std::sys_common::backtrace::_print_fmt::h08c31be18fedf422
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/sys_common/backtrace.rs:67:5
     3:     0xaaaaaf4a5d98 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc38bcf44d9e857e3
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/sys_common/backtrace.rs:44:22
     4:     0xaaaaaf4c7400 - core::fmt::rt::Argument::fmt::ha5b752f9cd7ef4a3
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/fmt/rt.rs:138:9
     5:     0xaaaaaf4c7400 - core::fmt::write::h9fac187ae7486f3c
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/fmt/mod.rs:1094:21
     6:     0xaaaaaf4a2d9c - std::io::Write::write_fmt::h239e9fb6296b3a7f
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/io/mod.rs:1714:15
     7:     0xaaaaaf4a5bcc - std::sys_common::backtrace::_print::h52f67cfa8753b0ab
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/sys_common/backtrace.rs:47:5
     8:     0xaaaaaf4a5bcc - std::sys_common::backtrace::print::hdea7481e2c957a93
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/sys_common/backtrace.rs:34:9
     9:     0xaaaaaf4a745c - std::panicking::default_hook::{{closure}}::h7c36fa733369c49e
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:270:22
    10:     0xaaaaaf4a7184 - std::panicking::default_hook::h303eee75f9a8f6a8
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:290:9
    11:     0xaaaaaf4a7a20 - std::panicking::rust_panic_with_hook::h270c94381ec34744
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:707:13
    12:     0xaaaaaf4a78e8 - std::panicking::begin_panic_handler::{{closure}}::h3653e3502bcc1625
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:599:13
    13:     0xaaaaaf4a627c - std::sys_common::backtrace::__rust_end_short_backtrace::h6b8510f2f024eeeb
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/sys_common/backtrace.rs:170:18
    14:     0xaaaaaf4a7638 - rust_begin_unwind
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
    15:     0xaaaaaf42c178 - core::panicking::panic_fmt::ha96945d7a1b20293
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
    16:     0xaaaaaf43a0b4 - cmake::fail::h62955ca510aa61ba
                                 at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5
    17:     0xaaaaaf439c58 - cmake::run::h2532d3f6b77168fd
                                 at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1068:13
    18:     0xaaaaaf436508 - cmake::Config::build::hcbc14bd8459bebb2
                                 at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:813:13
    19:     0xaaaaaf42e2e8 - build_script_build_zng::build_zlib_ng::h0ddfdc542c1d0d30
                                 at /cargo/registry/src/index.crates.io-6f17d22bba15001f/libz-ng-sys-1.1.12/build_zng.rs:21:23
    20:     0xaaaaaf42eb38 - build_script_build_zng::main::h15801808408216ce
                                 at /cargo/registry/src/index.crates.io-6f17d22bba15001f/libz-ng-sys-1.1.12/build_zng.rs:59:5
    21:     0xaaaaaf42cc64 - core::ops::function::FnOnce::call_once::h92a9a46ca5c1b2ba
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:250:5
    22:     0xaaaaaf42c7a8 - std::sys_common::backtrace::__rust_begin_short_backtrace::hf040af9fc854e094
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/sys_common/backtrace.rs:154:18
    23:     0xaaaaaf42c93c - std::rt::lang_start::{{closure}}::hc4ae6191443b941f
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:166:18
    24:     0xaaaaaf49ec04 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h88e11b183391c045
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:284:13
    25:     0xaaaaaf49ec04 - std::panicking::try::do_call::hf62418fc1f7ab0ba
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40
    26:     0xaaaaaf49ec04 - std::panicking::try::h96e06e195e92b5b2
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19
    27:     0xaaaaaf49ec04 - std::panic::catch_unwind::h81d0af01faa937a6
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14
    28:     0xaaaaaf49ec04 - std::rt::lang_start_internal::{{closure}}::hf32b9834747c984e
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:48
    29:     0xaaaaaf49ec04 - std::panicking::try::do_call::had623a98a0d3cc26
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40
    30:     0xaaaaaf49ec04 - std::panicking::try::ha216a98a719bb195
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19
    31:     0xaaaaaf49ec04 - std::panic::catch_unwind::h47218f3906a674ae
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14
    32:     0xaaaaaf49ec04 - std::rt::lang_start_internal::h9020369f9618c61f
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:20
    33:     0xaaaaaf42c90c - std::rt::lang_start::hba026c9cf1629a70
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:165:17
    34:     0xaaaaaf42eb90 - main
    35:     0xffff98cb7780 - <unknown>
    36:     0xffff98cb7858 - __libc_start_main
    37:     0xaaaaaf42c6b0 - _start
    38:                0x0 - <unknown>
warning: build failed, waiting for other jobs to finish...
root@b643aef843f5:/usr/local/src/deno# ls
Cargo.lock  Cargo.toml  LICENSE.md  README.md  Releases.md  bench_util  cli  ext  runtime  rust-toolchain.toml  target  test_ffi  test_napi  test_util  tools
Byron commented 1 year ago

It appears the log previously was truncated, and now it seems complete claiming that cmake isn't installed. If the dockerfile is any indicator, that may indeed be the case.

  --- stderr
  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  failed to execute command: No such file or directory (os error 2)
  is `cmake` not installed?

Can you try to install cmake see where that gets us?

tackn77 commented 1 year ago

Thank you very much. I was able to complete the build.

I added the following

RUN apt-get update && apt-get install -y cmake protobuf-compiler

The previous log will be the one that was not output where it entered the container and passed after the second time.

I apologize for the trouble, as I was able to build without adding it before, so I don't know what is wrong.