exodus bundling fail silently on docker build

py023 commented 9 months ago

Enviroment:

Arch: arm64
Version: v0.19

Issue description:

I tried to use Dockerfile to generate a ldb_toolchain_gen.sh for arm64. It worked fine in ldb-toolchain v0.18. But in v0.19, I built docker image succesfully then docker run --rm -v /tmp/some_toolchain_gen:/data ldbtoolchain:arm64-ldb-19 failed with cp: cannot stat '/opt/exodus/bundles/*/lib/aarch64-linux-gnu/*': No such file or directory.

Additional information

I attached into the container, there was no /opt/exodus folder in container:

# ls /opt/
cmake-3.22.1-linux-aarch64.tar.gz  libc6_2.27-3ubuntu1.6_arm64.deb  patchelf-0.14.3-aarch64.tar.gz  printers.py

Looks like it's because RUN exodus failed but didn't report failure. Then I manually executed the exodus xxx ... | bash command, it failed sliently as well:

# exodus /usr/bin/yasm /usr/bin/nasm /usr/bin/nm /usr/bin/addr2line /usr/bin/python3 /usr/bin/curl /usr/bin/gdb /usr/bin/ninja     /usr/bin/m4 /usr/bin/bison /usr/bin/yacc /usr/bin/flex /usr/bin/pkg-config /usr/bin/as /usr/bin/ld.bfd     /usr/bin/gcc-ranlib-${GCC_VERSION} /usr/bin/g++-${GCC_VERSION} /usr/bin/gcc-ar-${GCC_VERSION}     /usr/bin/gcc-nm-${GCC_VERSION}     /usr/bin/gcc-${GCC_VERSION}     /usr/bin/${ARCH}-linux-gnu-cpp-${GCC_VERSION}     /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/lto1     /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/lto-wrapper     /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/cc1     /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/cc1plus     /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/collect2     /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/g++-mapper-server     /usr/bin/lldb-argdumper-${LLVM_VERSION}     /usr/bin/lldb-instr-${LLVM_VERSION}     /usr/bin/lldb-server-${LLVM_VERSION}     /usr/bin/lldb-vscode-${LLVM_VERSION}     /usr/bin/lldb-${LLVM_VERSION}     /usr/bin/clangd-${LLVM_VERSION}     /usr/bin/clang-tidy-${LLVM_VERSION}     /usr/bin/clang-format-${LLVM_VERSION}     /usr/bin/clang-cpp-${LLVM_VERSION}     /usr/bin/clang-${LLVM_VERSION}     /usr/lib/llvm-${LLVM_VERSION}/bin/llvm-link     /usr/lib/llvm-${LLVM_VERSION}/bin/llc     /usr/lib/llvm-${LLVM_VERSION}/bin/opt     /usr/bin/llvm-strip-${LLVM_VERSION}     /usr/bin/llvm-install-name-tool-${LLVM_VERSION}     /usr/bin/llvm-objcopy-${LLVM_VERSION}     /usr/bin/llvm-ranlib-${LLVM_VERSION}     /usr/bin/llvm-ar-${LLVM_VERSION}     /usr/bin/llvm-nm-${LLVM_VERSION}     /usr/bin/llvm-cov-${LLVM_VERSION}     /usr/bin/llvm-profdata-${LLVM_VERSION}     /usr/bin/llvm-profgen-${LLVM_VERSION}     /usr/bin/lld-${LLVM_VERSION} | bash
# echo "code $? "
code 0

I know nothing about exodus so I blindly bitsect the arguments, only command below reports error:(not sure if related)

exodus /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/lto1 \
    /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/lto-wrapper \
    /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/cc1 \
    /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/cc1plus \
    /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/collect2 \
    /usr/libexec/gcc/${ARCH}-linux-gnu/${GCC_VERSION}/g++-mapper-server | bash

Installing executable bundle in "/opt/exodus"...

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Something went wrong, please send an email to contact@intoli.com with details about the bundle.

Without gcc tool above, exodus handles bundling just fine.

amosbird commented 5 months ago

Hi. Apologies for the delay in response. I currently lack an arm64 environment to reproduce. Ideally we should identify dependencies by ourselves instead of blindly trusting Exodus. I will work on this in the next release.

yixi-gu commented 3 months ago

I have the same problem use master branch. does it fix？

amosbird commented 3 months ago

Now the Exodus tool has been removed and I've published an aarch64 toolchain with LLVM-17. This change was necessary because LLVM-18 is broken in the upstream Ubuntu 18 aarch64 environment.

amosbird / ldb_toolchain_gen