search

NVIDIA / warp

A Python framework for high performance GPU simulation and graphics
https://nvidia.github.io/warp/
Other
1.75k stars 148 forks source link

Warp Clang/LLVM build error when python build_lib.py #166

Closed Leon-LXA closed 2 months ago

Leon-LXA commented 5 months ago

I am building warp.so by "python build_lib.py --cuda_path="/usr/local/cuda-12" ", however it has problem:

Namespace(build_llvm=False, cuda_path='/usr/local/cuda-12/', debug_llvm=False, fast_math=False, mode='release', msvc_path=None, quick=False, sdk_path=None, standalone=True, verbose=True, verify_fp=False) Finished writing /home/leon/origin/warp/warp/native/exports.h Building /home/leon/origin/warp/warp/bin/warp.so g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/warp.cpp" -o "/home/leon/origin/warp/warp/native/warp.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/crt.cpp" -o "/home/leon/origin/warp/warp/native/crt.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/cuda_util.cpp" -o "/home/leon/origin/warp/warp/native/cuda_util.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/mesh.cpp" -o "/home/leon/origin/warp/warp/native/mesh.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/hashgrid.cpp" -o "/home/leon/origin/warp/warp/native/hashgrid.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/reduce.cpp" -o "/home/leon/origin/warp/warp/native/reduce.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/runlength_encode.cpp" -o "/home/leon/origin/warp/warp/native/runlength_encode.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/sort.cpp" -o "/home/leon/origin/warp/warp/native/sort.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/sparse.cpp" -o "/home/leon/origin/warp/warp/native/sparse.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/volume.cpp" -o "/home/leon/origin/warp/warp/native/volume.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/marching.cpp" -o "/home/leon/origin/warp/warp/native/marching.cpp.o" g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/cutlass_gemm.cpp" -o "/home/leon/origin/warp/warp/native/cutlass_gemm.cpp.o" build took 10073.37 ms "/usr/local/cuda-12//bin/nvcc" -O3 --compiler-options -fPIC,-fvisibility=hidden -gencode=arch=compute_52,code=sm_52 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=sm_86 -gencode=arch=compute_89,code=sm_89 -gencode=arch=compute_90,code=sm_90 -gencode=arch=compute_90,code=compute_90 -t0 --extended-lambda -DNDEBUG -DWP_ENABLE_CUDA=1 -I"/home/leon/origin/warp/warp/native" -DWP_ENABLE_CUTLASS=1 -I"warp/native/cutlass/include" -I"warp/native/cutlass/tools/util/include" -o "/home/leon/origin/warp/warp/native/warp.cu.o" -c "/home/leon/origin/warp/warp/native/warp.cu" /home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used const void* src_grad = ^

Remark: The warnings can be suppressed with "-diag-suppress "

/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used void* dst_grad = ^

build_cuda took 92215.20 ms g++ -shared -Wl,-rpath,'$ORIGIN' -Wl,--no-undefined -Wl,--exclude-libs,ALL -o '/home/leon/origin/warp/warp/bin/warp.so' "/home/leon/origin/warp/warp/native/warp.cpp.o" "/home/leon/origin/warp/warp/native/crt.cpp.o" "/home/leon/origin/warp/warp/native/cuda_util.cpp.o" "/home/leon/origin/warp/warp/native/mesh.cpp.o" "/home/leon/origin/warp/warp/native/hashgrid.cpp.o" "/home/leon/origin/warp/warp/native/reduce.cpp.o" "/home/leon/origin/warp/warp/native/runlength_encode.cpp.o" "/home/leon/origin/warp/warp/native/sort.cpp.o" "/home/leon/origin/warp/warp/native/sparse.cpp.o" "/home/leon/origin/warp/warp/native/volume.cpp.o" "/home/leon/origin/warp/warp/native/marching.cpp.o" "/home/leon/origin/warp/warp/native/cutlass_gemm.cpp.o" "/home/leon/origin/warp/warp/native/warp.cu.o" -L"/usr/local/cuda-12//lib64" -lcudart_static -lnvrtc_static -lnvrtc-builtins_static -lnvptxcompiler_static -lpthread -ldl -lrt strip --strip-all --keep-symbol=__jit_debug_register_code --keep-symbol=__jit_debug_descriptor /home/leon/origin/warp/warp/bin/warp.so link took 1623.12 ms Fetching packman-common@7.5.zip from bootstrap.packman.nvidia.com ... Unpacking ... Warp Clang/LLVM build error: run build.bat / build.sh

shi-eric commented 5 months ago

Looking at build_llvm.py, it seems like that message got printed because _build/host-deps/llvm-project wasn't successfully created. What happens if you run

./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose

@c0d1f1ed : We should change that message since build.sh is no longer there, right?

Leon-LXA commented 5 months ago

Looking at build_llvm.py, it seems like that message got printed because _build/host-deps/llvm-project wasn't successfully created. What happens if you run

./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose

@c0d1f1ed : We should change that message since build.sh is no longer there, right?

still the same error. And I do not find any path as "_build/host-deps/llvm-project"

shi-eric commented 5 months ago

Looking at build_llvm.py, it seems like that message got printed because _build/host-deps/llvm-project wasn't successfully created. What happens if you run

./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose

@c0d1f1ed : We should change that message since build.sh is no longer there, right?

still the same error. And I do not find any path as "_build/host-deps/llvm-project"

Surely you get a different error when you just run the packman command directly, no?

Leon-LXA commented 5 months ago

Looking at build_llvm.py, it seems like that message got printed because _build/host-deps/llvm-project wasn't successfully created. What happens if you run

./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose

@c0d1f1ed : We should change that message since build.sh is no longer there, right?

still the same error. And I do not find any path as "_build/host-deps/llvm-project"

Surely you get a different error when you just run the packman command directly, no?

./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose Fetching packman-common@7.5.zip from bootstrap.packman.nvidia.com ... Unpacking ...

Leon-LXA commented 5 months ago

Looking at build_llvm.py, it seems like that message got printed because _build/host-deps/llvm-project wasn't successfully created. What happens if you run

./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose

@c0d1f1ed : We should change that message since build.sh is no longer there, right?

still the same error. And I do not find any path as "_build/host-deps/llvm-project"

Surely you get a different error when you just run the packman command directly, no?

My two computers act differently. This one just unpacking and finished. And the error's the same when i python build_lib.py. The other one unpacking and does many things, then no error when python build_lib.py

shi-eric commented 5 months ago

Thanks! It's hard to say exactly what's wrong with your setup but here's some thoughts. Packman needs to install "itself" somewhere on your system, and the environment variable PM_PACKAGES_ROOT controls the location. By default, it picks /home/eshi/.cache/packman/ on my system, and in that I have the directories packman-common, python, and chk. You can try setting PM_PACKAGES_ROOT to a different directory that you can write to...

Alternatively, you can also try to manually run what the install process is trying to do... Download http://bootstrap.packman.nvidia.com/packman-common@7.5.zip onto your system, then run python3 tools/packman/bootstrap/install_package.py packman-common@7.5.zip $PM_PACKAGES_ROOT/packman-common/7.5. Then you can try running ./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 again.

c0d1f1ed commented 2 months ago

I'm going to close this for now, but please reopen it or file a new bug if this issue still reproduces. The code that used to potentially produce this error has changed a fair bit in recent months so we'd need updated information anyway.

If you do run into this still, it might be worth trying to clone the Warp repo into a new directory and build from there. Interrupted downloads or builds could cause issues we currently don't detect and report properly.