weld-project / weld

High-performance runtime for data analytics applications
https://www.weld.rs
BSD 3-Clause "New" or "Revised" License
2.99k stars 260 forks source link

Build error while on Fedora #512

Open ConcurrencyPractitioner opened 4 years ago

ConcurrencyPractitioner commented 4 years ago

While running the python setup.py develop command under weld-python on Fedora. We found the following error:

Fresh uuid v0.7.4
       Fresh llvm-sys v60.4.1
   Compiling weld v0.4.0 (/home/zyu/weld/weld)
     Running `/home/zyu/weld/weld-python/target/debug/build/weld-6c83ee8deea2cb14/build-script-build`
error: failed to run custom build command for `weld v0.4.0 (/home/zyu/weld/weld)`

Caused by:
  process didn't exit successfully: `/home/zyu/weld/weld-python/target/debug/build/weld-6c83ee8deea2cb14/build-script-build` (exit code: 101)
--- stdout
cargo:rustc-env=BUILD_ID=dcbba9a

cargo:rustc-link-lib=dylib=stdc++
make: Entering directory '/home/zyu/weld/weld/llvmext'
clang++-9.0 -O3 -fno-use-cxa-atexit -I/usr/include -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/usr/include -std=c++11   -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -c llvmext.cpp -o /home/zyu/weld/weld-python/target/debug/build/weld-0807948382241f60/out/llvmext.o
make: Leaving directory '/home/zyu/weld/weld/llvmext'

--- stderr
make: clang++-9.0: Command not found
make: *** [Makefile:26: llvmext.o] Error 127
thread 'main' panicked at 'assertion failed: status.success()', /home/zyu/weld/weld/build.rs:41:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

warning: build failed, waiting for other jobs to finish...
ConcurrencyPractitioner commented 4 years ago

After a little bit of tinkering (installing rustup and rust), we ended up with this error:

error: linking with `cc` failed: exit code: 1
...
  = note: /usr/bin/ld: /home/zyu/weld/weld-python/target/debug/deps/libweld-13840bb6bd71b796.rlib(llvmext.o): relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: final link failed: nonrepresentable section on output
          collect2: error: ld returned 1 exit status

error: aborting due to previous error

error: could not compile `weld-python`.

Caused by:
  process didn't exit successfully: `rustc --crate-name weld_python --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type cdylib --emit=dep-info,link -Cbitcode-in-rlib=no -C debuginfo=2 --crate-type cdylib -C metadata=186a4106a799954a --out-dir /home/zyu/weld/weld-python/target/debug/deps -C incremental=/home/zyu/weld/weld-python/target/debug/incremental -L dependency=/home/zyu/weld/weld-python/target/debug/deps --extern libc=/home/zyu/weld/weld-python/target/debug/deps/liblibc-7f3b8e4c2db2a8d8.rlib --extern pyo3=/home/zyu/weld/weld-python/target/debug/deps/libpyo3-616294b0595564e1.rlib --extern weld=/home/zyu/weld/weld-python/target/debug/deps/libweld-13840bb6bd71b796.rlib -C link-args=-Wl,-export-dynamic -L native=/home/zyu/weld/weld-python/target/debug/build/weld-09890efcb0f540af/out -L native=/home/zyu/weld/weld-python/target/debug/build/llvm-sys-fc86408c0fa835a8/out -L native=/usr/lib64` (exit code: 1)
error: cargo failed with code: 101

During the tinkering, we also sym-linked "pyo3-pack" to "pyo3".

twitu commented 2 years ago

I'm facing a similar issue. I'm trying to run some of the weld tests using cargo test --package weld --test binop_tests -- simple_binop --exact but it's giving linking errors.

= note: /usr/bin/ld: /home/twitu/Code/weld/target/debug/deps/libweld-8a5fb61350185090.rlib(llvmext.o): relocation R_X86_64_32 against `.bss' can not be used when making a PIE object; recompile with -fPIE

I've checked the following -

Make sure llvm-config is found on $PATH and llvm-config --version gives back 5.0.0 Try adding the LLVM libraries to the LD_LIBRARY_PATH (or whatever the Fedora equivalent is) The version of the llvm-sys crate you use must match the LLVM version you are using. Make sure you change llvm-sys = "38.0.1" to llvm-sys = "50.0.0" in Cargo.toml.

Although now it's difficult to find and install llvm 6.0.0 so instead I'm using llvm version 10.0.0 and llvm-sys version in Cargo.toml is "100.2.0". Still I don't think the linking error is because of version difference, because when I add -static to the raw command it seems to work.

 "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/twitu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/twitu/Code/weld/target/debug/deps/binop_tests-defb0983ccab1308" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-Wl,-export-dynamic" **"-static"**

It'll be a big help if someone could share how they overcame this issue. Or how to pass the extra -static flag through the test harness, currently I'm just adding to the raw command shown in the error. After adding -static the command succeeds but the rest of steps for the test are no longer executed.