search

rust-lang / libz-sys

Rust crate package to link to a system libz (zlib)
Apache License 2.0
110 stars 73 forks source link

Add missing package files #195

Closed Jake-Shadle closed 1 month ago

Jake-Shadle commented 1 month ago

Resolves: #194

Byron commented 1 month ago

Thanks for your help!

Submodules are checked out on CI, yet the hardcoded file seems to be missing. I checked and it's available locally, strangely enough.

When trying the package commands locally, I also see two failures, both with zlib-ng and zlib-ng-cc. The build of zlib-ng-cc also fails for me, but works for zlib-ng.

I have a feeling there is a CWD difference between what cargo package does and what cargo build does - maybe one should use environment variables to get the manifest-root instead of relying on the CWD.

Here are my logs for completeness.

Logs ``` ❯ cargo package --no-default-features --features zlib-ng-no-cmake-experimental-community-maintained Packaging libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Verifying libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Updating crates.io index Compiling cc v1.0.97 Compiling libc v0.2.153 Compiling pkg-config v0.3.30 Compiling vcpkg v0.2.15 Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18) The following warnings were emitted during compilation: warning: libz-sys@1.1.18: Compiler family detection failed due to error: IOError: No such file or directory (os error 2) error: failed to run custom build command for `libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18)` Caused by: process didn't exit successfully: `/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-7889a033ca7521fc/build-script-build` (exit status: 101) --- stdout cargo:rerun-if-env-changed=LIBZ_SYS_STATIC cargo:rerun-if-changed=build.rs cargo:rerun-if-changed=zng/cmake.rs cargo:rerun-if-changed=zng/cc.rs OPT_LEVEL = Some("0") TARGET = Some("aarch64-apple-darwin") HOST = Some("aarch64-apple-darwin") cargo:rerun-if-env-changed=CC_aarch64-apple-darwin CC_aarch64-apple-darwin = None cargo:rerun-if-env-changed=CC_aarch64_apple_darwin CC_aarch64_apple_darwin = None cargo:rerun-if-env-changed=HOST_CC HOST_CC = None cargo:rerun-if-env-changed=CC CC = None cargo:warning=Compiler family detection failed due to error: IOError: No such file or directory (os error 2) cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS CRATE_CC_NO_DEFAULTS = None DEBUG = Some("true") CARGO_CFG_TARGET_FEATURE = Some("aes,crc,dit,dotprod,dpb,dpb2,fcma,fhm,flagm,fp16,frintts,jsconv,lor,lse,neon,paca,pacg,pan,pmuv3,ras,rcpc,rcpc2,rdm,sb,sha2,sha3,ssbs,vh") cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT cargo:rerun-if-env-changed=CFLAGS_aarch64-apple-darwin CFLAGS_aarch64-apple-darwin = None cargo:rerun-if-env-changed=CFLAGS_aarch64_apple_darwin CFLAGS_aarch64_apple_darwin = None cargo:rerun-if-env-changed=HOST_CFLAGS HOST_CFLAGS = None cargo:rerun-if-env-changed=CFLAGS CFLAGS = None --- stderr thread 'main' panicked at zng/cc.rs:65:10: called `Result::unwrap()` on an `Err` value: "failed to read \"src/zlib-ng/gzread.c.in\": No such file or directory (os error 2)" note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace error: failed to verify package tarball libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] ❯ cargo package --no-default-features --features zlib-ng Packaging libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Verifying libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Updating crates.io index Compiling cc v1.0.97 Compiling libc v0.2.153 Compiling vcpkg v0.2.15 Compiling pkg-config v0.3.30 Compiling cmake v0.1.50 Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18) error: failed to run custom build command for `libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18)` Caused by: process didn't exit successfully: `/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-0303a1ebf4c2e3d8/build-script-build` (exit status: 101) --- stdout cargo:rerun-if-env-changed=LIBZ_SYS_STATIC cargo:rerun-if-changed=build.rs cargo:rerun-if-changed=zng/cmake.rs cargo:rerun-if-changed=zng/cc.rs CMAKE_TOOLCHAIN_FILE_aarch64-apple-darwin = None CMAKE_TOOLCHAIN_FILE_aarch64_apple_darwin = None HOST_CMAKE_TOOLCHAIN_FILE = None CMAKE_TOOLCHAIN_FILE = None CMAKE_GENERATOR_aarch64-apple-darwin = None CMAKE_GENERATOR_aarch64_apple_darwin = None HOST_CMAKE_GENERATOR = None CMAKE_GENERATOR = None CMAKE_PREFIX_PATH_aarch64-apple-darwin = None CMAKE_PREFIX_PATH_aarch64_apple_darwin = None HOST_CMAKE_PREFIX_PATH = None CMAKE_PREFIX_PATH = None CMAKE_aarch64-apple-darwin = None CMAKE_aarch64_apple_darwin = None HOST_CMAKE = None CMAKE = None running: cd "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-db60ce2f2aa8146d/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/src/zlib-ng" "-DCMAKE_OSX_ARCHITECTURES=arm64" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=ON" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_INSTALL_PREFIX=/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-db60ce2f2aa8146d/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=14.5" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=14.5" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=14.5" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Debug" --- stderr CMake Warning: Ignoring extra path from command line: "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/src/zlib-ng" CMake Error: The source directory "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/src/zlib-ng" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI. thread 'main' panicked at /Users/byron/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5: command did not execute successfully, got: exit status: 1 build script failed, must exit now note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace error: failed to verify package tarball libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] took 2s ❯ l src/zlib-ng/CMakeLists.txt libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] ❯ cargo build Compiling libc v0.2.153 Compiling cc v1.0.90 Compiling vcpkg v0.2.15 Compiling pkg-config v0.3.30 Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.78s libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] ❯ cargo build --all-features Compiling cmake v0.1.50 Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.93s libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] ❯ libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] ❯ cargo build --no-default-features --features zlib-ng-no-cmake-experimental-community-maintained Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) The following warnings were emitted during compilation: warning: libz-sys@1.1.18: In file included from src/zlib-ng/arch/arm/adler32_neon.c:9: warning: libz-sys@1.1.18: src/zlib-ng/arch/arm/neon_intrins.h:31:28: error: expected identifier or '(' warning: libz-sys@1.1.18: static inline uint16x8x4_t vld1q_u16_x4(uint16_t const *a) { warning: libz-sys@1.1.18: ^ warning: libz-sys@1.1.18: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10524:28: note: expanded from macro 'vld1q_u16_x4' warning: libz-sys@1.1.18: #define vld1q_u16_x4(__p0) __extension__ ({ \ warning: libz-sys@1.1.18: ^ warning: libz-sys@1.1.18: In file included from src/zlib-ng/arch/arm/adler32_neon.c:9: warning: libz-sys@1.1.18: src/zlib-ng/arch/arm/neon_intrins.h:40:28: error: expected identifier or '(' warning: libz-sys@1.1.18: static inline uint8x16x4_t vld1q_u8_x4(uint8_t const *a) { warning: libz-sys@1.1.18: ^ warning: libz-sys@1.1.18: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10467:27: note: expanded from macro 'vld1q_u8_x4' warning: libz-sys@1.1.18: #define vld1q_u8_x4(__p0) __extension__ ({ \ warning: libz-sys@1.1.18: ^ warning: libz-sys@1.1.18: In file included from src/zlib-ng/arch/arm/adler32_neon.c:9: warning: libz-sys@1.1.18: src/zlib-ng/arch/arm/neon_intrins.h:49:20: error: expected identifier or '(' warning: libz-sys@1.1.18: static inline void vst1q_u16_x4(uint16_t *p, uint16x8x4_t a) { warning: libz-sys@1.1.18: ^ warning: libz-sys@1.1.18: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:28042:34: note: expanded from macro 'vst1q_u16_x4' warning: libz-sys@1.1.18: #define vst1q_u16_x4(__p0, __p1) __extension__ ({ \ warning: libz-sys@1.1.18: ^ warning: libz-sys@1.1.18: 3 errors generated. error: failed to run custom build command for `libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)` Caused by: process didn't exit successfully: `/Users/byron/dev/github.com/rust-lang/libz-sys/target/debug/build/libz-sys-6de51d426d3975ac/build-script-build` (exit status: 101) --- stdout cargo:rerun-if-env-changed=LIBZ_SYS_STATIC cargo:rerun-if-changed=build.rs cargo:rerun-if-changed=zng/cmake.rs cargo:rerun-if-changed=zng/cc.rs OPT_LEVEL = Some("0") TARGET = Some("aarch64-apple-darwin") HOST = Some("aarch64-apple-darwin") cargo:rerun-if-env-changed=CC_aarch64-apple-darwin CC_aarch64-apple-darwin = None cargo:rerun-if-env-changed=CC_aarch64_apple_darwin CC_aarch64_apple_darwin = None cargo:rerun-if-env-changed=HOST_CC HOST_CC = None cargo:rerun-if-env-changed=CC CC = None cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS CRATE_CC_NO_DEFAULTS = None DEBUG = Some("true") cargo:rerun-if-env-changed=CFLAGS_aarch64-apple-darwin CFLAGS_aarch64-apple-darwin = None cargo:rerun-if-env-changed=CFLAGS_aarch64_apple_darwin CFLAGS_aarch64_apple_darwin = None cargo:rerun-if-env-changed=HOST_CFLAGS HOST_CFLAGS = None cargo:rerun-if-env-changed=CFLAGS CFLAGS = None cargo:warning=In file included from src/zlib-ng/arch/arm/adler32_neon.c:9: cargo:warning=src/zlib-ng/arch/arm/neon_intrins.h:31:28: error: expected identifier or '(' cargo:warning=static inline uint16x8x4_t vld1q_u16_x4(uint16_t const *a) { cargo:warning= ^ cargo:warning=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10524:28: note: expanded from macro 'vld1q_u16_x4' cargo:warning=#define vld1q_u16_x4(__p0) __extension__ ({ \ cargo:warning= ^ cargo:warning=In file included from src/zlib-ng/arch/arm/adler32_neon.c:9: cargo:warning=src/zlib-ng/arch/arm/neon_intrins.h:40:28: error: expected identifier or '(' cargo:warning=static inline uint8x16x4_t vld1q_u8_x4(uint8_t const *a) { cargo:warning= ^ cargo:warning=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10467:27: note: expanded from macro 'vld1q_u8_x4' cargo:warning=#define vld1q_u8_x4(__p0) __extension__ ({ \ cargo:warning= ^ cargo:warning=In file included from src/zlib-ng/arch/arm/adler32_neon.c:9: cargo:warning=src/zlib-ng/arch/arm/neon_intrins.h:49:20: error: expected identifier or '(' cargo:warning=static inline void vst1q_u16_x4(uint16_t *p, uint16x8x4_t a) { cargo:warning= ^ cargo:warning=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:28042:34: note: expanded from macro 'vst1q_u16_x4' cargo:warning=#define vst1q_u16_x4(__p0, __p1) __extension__ ({ \ cargo:warning= ^ cargo:warning=3 errors generated. --- stderr ToolExecError: Command env -u IPHONEOS_DEPLOYMENT_TARGET "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-2" "-fno-omit-frame-pointer" "--target=arm64-apple-darwin" "-mmacosx-version-min=14.5" "-std=c11" "-I" "/Users/byron/dev/github.com/rust-lang/libz-sys/target/debug/build/libz-sys-cf968d9e59cebf55/out/include" "-I" "src/zlib-ng" "-Werror-implicit-function-declaration" "-fvisibility=hidden" "-march=armv8-a+crc+simd" "-DZLIB_COMPAT" "-DWITH_GZFILEOP" "-DHAVE_ALIGNED_ALLOC" "-DHAVE_ATTRIBUTE_ALIGNED" "-DHAVE_BUILTIN_CTZ" "-DHAVE_BUILTIN_CTZLL" "-DHAVE_THREAD_LOCAL" "-DHAVE_VISIBILITY_HIDDEN" "-DHAVE_VISIBILITY_INTERNAL" "-D_LARGEFILE64_SOURCE=1" "-D__USE_LARGEFILE64" "-DSTDC" "-D_POSIX_SOURCE" "-DHAVE_POSIX_MEMALIGN" "-D_C99_SOURCE" "-DARM_FEATURES" "-DARM_ACLE" "-DHAVE_ARM_ACLE_H" "-DARM_NEON" "-o" "/Users/byron/dev/github.com/rust-lang/libz-sys/target/debug/build/libz-sys-cf968d9e59cebf55/out/lib/1b0507d0191b23d7-adler32_neon.o" "-c" "src/zlib-ng/arch/arm/adler32_neon.c" with args "cc" did not execute successfully (status code exit status: 1). thread 'main' panicked at zng/cc.rs:439:9: failed to compile zlib-ng with cc: detected compiler version as --- Apple clang version 15.0.0 (clang-1500.3.9.4) Target: arm64-apple-darwin23.4.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin --- note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] ❯ cargo build --no-default-features --features zlib-ng Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys) Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.01s ```
Jake-Shadle commented 1 month ago

Opened a PR on cc to fix the issue I found when debugging the packaging, but for now the workaround works. https://github.com/rust-lang/cc-rs/pull/1072

Byron commented 1 month ago

Thank you! It's very strange that CI is now failing, it's probably flakiness due to the nightly GNU compiler, but at least today it reproduces when retrying. Maybe the easiest is to try again tomorrow hoping it's fixed by then.

Byron commented 1 month ago

Maybe it's related to this change? The timing would match: https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html

NobodyXu commented 1 month ago

cc 1.0.98 has released

Jake-Shadle commented 1 month ago

Unfortunately the documented method of disabling either isn't functioning or there is something else going on. See also https://github.com/cross-rs/cross/issues/1496. It might be related to the LLD change, but if it is there is something wrong as targets other than x86_64-unknown-linux-gnu are also affected, which shouldn't be the case according to the post.

Byron commented 1 month ago

I see, thanks for letting me know. Maybe a solution arises in the coming days/weeks, so we can finish this fix and finally bring CC-zlib-ng to the people :).

Jake-Shadle commented 1 month ago

Upstream rust issue https://github.com/rust-lang/rust/issues/125330

Jake-Shadle commented 1 month ago

The nightly upstream fix works and I removed the cc workaround.

Byron commented 1 month ago

Fantastic, let me create a new build right away, let's see if it works.