OlivierDehaene
commented
8 months ago What version of nvcc and gcc do you have?
Open gctian opened 8 months ago
OlivierDehaene
commented
8 months ago What version of nvcc and gcc do you have?
gctian
commented
8 months ago What version of nvcc and gcc do you have?
ikalista
commented
7 months ago I have a very evil method you can say try it, I'm getting a recompile with -fPIC error, then I solved it by doing this #59 .
RUN: find / -name build.rs | grep candle-flash-attn, then you can find the build code for candle-flash-attn.
In the right place add: .arg("--compiler-options") .arg("-fPIE")
cargo clean and rebuild
gctian
commented
7 months ago I have a very evil method you can say try it, I'm getting a recompile with -fPIC error, then I solved it by doing this #59 .
RUN: find / -name build.rs | grep candle-flash-attn, then you can find the build code for candle-flash-attn.
In the right place add: .arg("--compiler-options") .arg("-fPIE")
cargo clean and rebuild
Thanks, I tried, but still got the error, and the error message this time has become very short.
error: linking with `cc` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/home/work/.rustup/toolchains/1.75.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/data/miniconda3/envs/search/bin:/usr/local/cuda/bin:/data/miniconda3/envs/search/bin:/home/work/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/remote-cli:/usr/local/cuda/bin:/home/work/.cargo/bin:/data/miniconda3/condabin:/usr/local/cuda/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/work/bin:/home/work/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustc7lX7Vv/symbols.o" "/home/work/search/text-embeddings-inference/target/release/deps/text_embeddings_router-aedef450ae8423fb.text_embeddings_router-cd21587c1b6864dd.46b75xtg3t69j2xs.rcgu.o.rcgu.o" "-Wl,--as-needed" "-L" "/home/work/search/text-embeddings-inference/target/release/deps" "-L" "/usr/local/openssl/lib" "-L" "/home/work/search/text-embeddings-inference/target/release/build/ring-4013101abfe8aaa6/out" "-L" "/home/work/search/text-embeddings-inference/target/release/build/libmimalloc-sys-bbc5687bef7082a2/out" "-L" "/usr/local/cuda/lib64" "-L" "/usr/local/cuda/lib64/stubs" "-L" "/usr/local/cuda/targets/x86_64-linux" "-L" "/usr/local/cuda/targets/x86_64-linux/lib" "-L" "/usr/local/cuda/targets/x86_64-linux/lib/stubs" "-L" "/home/work/search/text-embeddings-inference/target/release/build/candle-flash-attn-894bcc1066d6869e/out" "-L" "/home/work/search/text-embeddings-inference/target/release/build/candle-layer-norm-2be8037f8b590827/out" "-L" "/home/work/search/text-embeddings-inference/target/release/build/candle-rotary-29f1133020e96866/out" "-L" "/home/work/search/text-embeddings-inference/target/release/build/esaxx-rs-43332f15536379a3/out" "-L" "/home/work/search/text-embeddings-inference/target/release/build/onig_sys-d79b1ed81dfb3450/out" "-L" "/home/work/.rustup/toolchains/1.75.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustc7lX7Vv/libesaxx_rs-0837297a42d47e28.rlib" "/tmp/rustc7lX7Vv/libonig_sys-ec90d3a8fb007da0.rlib" "/tmp/rustc7lX7Vv/libring-f6f9fd261d1c845a.rlib" "/tmp/rustc7lX7Vv/liblibmimalloc_sys-868ba60de6477d6b.rlib" "/home/work/.rustup/toolchains/1.75.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-8e138eaf26ebb4a8.rlib" "-Wl,-Bdynamic" "-lstdc++" "-lssl" "-lcrypto" "-lrotary" "-lcudart" "-lstdc++" "-llayernorm" "-lcudart" "-lstdc++" "-lflashattention" "-lcudart" "-lstdc++" "-lcuda" "-lnvrtc" "-lcurand" "-lcublas" "-lcublasLt" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/work/.rustup/toolchains/1.75.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/work/search/text-embeddings-inference/target/release/deps/text_embeddings_router-aedef450ae8423fb" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"
= note: /usr/bin/ld: /home/work/search/text-embeddings-inference/target/release/build/candle-rotary-29f1133020e96866/out/librotary.a(rotary-b6f3db7aa1741882.o): relocation R_X86_64_32 against symbol `_ZN4vllm23rotary_embedding_kernelIfLb0EEEvPT_S2_PKS1_S4_illiii' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /home/work/search/text-embeddings-inference/target/release/build/candle-layer-norm-2be8037f8b590827/out/liblayernorm.a(ln_api.o): relocation R_X86_64_32 against symbol `_ZN10layer_norm13ln_fwd_kernelINS_13Kernel_traitsIfffffjLj256ELj1ELj4ELj1ELj16ENS_18Kernel_traits_baseILj256EfffffjLj128EEEEELb1ELb1ELb1ELb1EEEvNS_9FwdParamsE' can not be used when making a PIE object; recompile with -fPIE
collect2: error: ld returned 1 exit status
error: could not compile `text-embeddings-router` (bin "text-embeddings-router") due to previous error
error: failed to compile `text-embeddings-router v1.1.0 (/home/work/search/text-embeddings-inference/router)`, intermediate artifacts can be found at `/home/work/search/text-embeddings-inference/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.This is my modification.
let builder = bindgen_cuda::Builder::default()
.kernel_paths(kernels)
.out_dir(build_dir.clone())
.arg("-std=c++17")
.arg("-O3")
.arg("--compiler-options")
.arg("-fPIC")
.arg("-U__CUDA_NO_HALF_OPERATORS__")
.arg("-U__CUDA_NO_HALF_CONVERSIONS__")
.arg("-U__CUDA_NO_HALF2_OPERATORS__")
.arg("-U__CUDA_NO_BFLOAT16_CONVERSIONS__")
.arg("-Icutlass/include")
.arg("--expt-relaxed-constexpr")
.arg("--expt-extended-lambda")
.arg("--use_fast_math")
.arg("--verbose");Fixed，also add two lines of code to the build.rs of candle-layer-norm 、candle-rotary solved the problem
OlivierDehaene
commented
7 months ago Your gcc version is very old and I think that's the rootcause but I'm honestly not sure.
ikalista
commented
7 months ago Your gcc version is very old and I think that's the rootcause but I'm honestly not sure.
I'm building from Dockerfile-cuda
The curl in the [nvidia/cuda:12.2.0-devel-ubuntu22.04] seemed to have problems in my environment, so I switched to [nvidia/cuda:12.3.2-devel-centos7]
Then I found out [nvidia/cuda:12.3.2-devel-centos7]'s gcc version is too low (about 4.8.5), how official image was build successfully from nvidia office images?
I'm still experiencing the compile tag issue above after manually upgrading gcc to 9.x.x, I still need to manually modify the build.rs file to make the build successful, honestly not sure what happened XD
System Info
GPU: A100 Python:3.10.13
Information
Tasks
Reproduction
cargo install --path router -F candle-cuda -F http --no-default-features，use this command to build.this is error message：
Thanks.
Expected behavior
build successful，and can use 'text-embeddings-router' cmd