peytoncai
commented
3 months ago same
Closed SleepyYui closed 3 months ago
peytoncai
commented
3 months ago same
HansvanHespen
commented
3 months ago I also have the problem that the build fails:
519.5 [93/93] : && /usr/bin/x86_64-linux-gnu-g++ -march=znver2 -std=gnu++20 -O3 -DNDEBUG vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llama-llava-cli.dir/llava-cli.cpp.o -o vendor/llama.cpp/examples/llava/llama-llava-cli -Wl,-rpath,/tmp/tmp56qdxi6v/build/vendor/llama.cpp/src:/tmp/tmp56qdxi6v/build/vendor/llama.cpp/ggml/src: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/src/libllama.so vendor/llama.cpp/ggml/src/libggml.so && :
519.5 **FAILED**: vendor/llama.cpp/examples/llava/llama-llava-cli
519.5 : && /usr/bin/x86_64-linux-gnu-g++ -march=znver2 -std=gnu++20 -O3 -DNDEBUG vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llama-llava-cli.dir/llava-cli.cpp.o -o vendor/llama.cpp/examples/llava/llama-llava-cli -Wl,-rpath,/tmp/tmp56qdxi6v/build/vendor/llama.cpp/src:/tmp/tmp56qdxi6v/build/vendor/llama.cpp/ggml/src: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/src/libllama.so vendor/llama.cpp/ggml/src/libggml.so && :
519.5 /usr/bin/ld: warning: libcuda.so.1, needed by vendor/llama.cpp/ggml/src/libggml.so, not found (try using -rpath or -rpath-link)
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemCreate'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemAddressReserve'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemUnmap'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemSetAccess'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuDeviceGet'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemAddressFree'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuGetErrorString'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuDeviceGetAttribute'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemMap'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemRelease'
519.5 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemGetAllocationGranularity'
519.5 collect2: error: ld returned 1 exit status
519.5 ninja: build stopped: subcommand failed.
519.6
519.6 *** CMake build failed
519.6 error: subprocess-exited-with-error"Working with the pre-built wheels is a bit unclear. I noticed there are three .so files in it, meaning three pre-compiled files. I would like to know a bit more how they are compiled. I have to cross-compile: compiling on my machine (i7 U8550), but for another machine on the cloud (AMD Epyc 7series). Hence the reason why I compile myself, with some very detailed options. It was difficult to configure CMake from the outside to make that work.
This is from my Dockerfile how I compile:
ENV FORCE_CMAKE=1
ENV CMAKE_ARGS="-DCMAKE_CUDA_COMPILER='/usr/local/cuda-12.2/bin/nvcc' -DCMAKE_CUDA_ARCHITECTURES='80' -DGGML_CUDA=ON -DGGML_NATIVE=OFF -DGGML_ALL_WARNINGS=OFF -DGGML_ACCELERATE=OFF"
ENV CXXFLAGS="-march=znver2 -std=gnu++20"
RUN pip install llama-cpp-python==0.2.86 --no-cache-dir --force-reinstall --upgrade --verboseVersion 0.2.33 compiles perfectly, but not version 0.2.86, which I need because 0.2.33 is not thread-safe and gives errors on my A100 40GB GPU.
If I compile without options, it compiles natively towards my i7 processor, and that gives a SIGINT on the Linux AMD Epyc machine in the cloud. (devel-Ubuntu 22.04)
Are the pre-compiled wheel files compiled using no processor features at all?
Anyway, compiling locally will lead to the most optimal executable, and it is a pity that it doesn't compile after all, in the latest step: 90/90. It seems to look for a libcuda.so.1 that it can't find to fuel libggml.so. Someone an idea?
High regards,
Hans
abetlen
commented
3 months ago @SleepyYui thank you for reporting, it seems the issue is the the vendor/llama.cpp/common/cmake folder is not being included in the pypi wheel?
Not sure at all why this would be, I'll look into this.
abetlen
commented
3 months ago @SleepyYui should be fixed now in 0.2.87
I've included the entire vendor/llama.cpp subdirectory in the source distribution as a workaround.
HansvanHespen
commented
3 months ago Dear @abetlen, as soon as the regular version 87 is out then, I will check as well.
gabrielmbmb
commented
3 months ago For some reason uv pip install llama-cpp-python==0.2.87 is failing too (it works for uv pip install llama-cpp-python==0.2.85):
uv pip install -U llama-cpp-python==0.2.87
⠧ llama-cpp-python==0.2.87 error: Failed to download and build `llama-cpp-python==0.2.87`
Caused by: Failed to extract archive
Caused by: failed to unpack `/Users/gabrielmbmb/Library/Caches/uv/built-wheels-v3/.tmpcTjUBT/llama_cpp_python-0.2.87/vendor/llama.cpp/spm-headers/ggml-alloc.h`
Caused by: File exists (os error 17) when symlinking ../ggml/include/ggml-alloc.h to /Users/gabrielmbmb/Library/Caches/uv/built-wheels-v3/.tmpcTjUBT/llama_cpp_python-0.2.87/vendor/llama.cpp/spm-headers/ggml-alloc.hI tried cleaning the cache (uv clean) but doesn't work. It works with pip install llama-cpp-python==0.2.87, so I think this could be a bug on uv side.
abetlen
commented
3 months ago @gabrielmbmb thank you for reporting this but I think that's a seperate issue specific to uv (I've opened a new issue for it #1670), a workaround is just to pip install which seems to handle the symlinks correctly.
I'm going to close this for now as it seems resolved.
HansvanHespen
commented
3 months ago Dear @abetlen and @gabrielmbmb,
Unfortunately, I still have the same error on 0.2.87 as well:
505.2 [93/93] : && /usr/bin/x86_64-linux-gnu-g++ -march=znver4 -std=gnu++23 -O3 -DNDEBUG vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llama-llava-cli.dir/llava-cli.cpp.o -o vendor/llama.cpp/examples/llava/llama-llava-cli -Wl,-rpath,/tmp/tmpaeyk5c4_/build/vendor/llama.cpp/src:/tmp/tmpaeyk5c4_/build/vendor/llama.cpp/ggml/src: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/src/libllama.so vendor/llama.cpp/ggml/src/libggml.so && :
505.2 **FAILED**: vendor/llama.cpp/examples/llava/llama-llava-cli
505.2 : && /usr/bin/x86_64-linux-gnu-g++ -march=znver4 -std=gnu++23 -O3 -DNDEBUG vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o vendor/llama.cpp/examples/llava/CMakeFiles/llama-llava-cli.dir/llava-cli.cpp.o -o vendor/llama.cpp/examples/llava/llama-llava-cli -Wl,-rpath,/tmp/tmpaeyk5c4_/build/vendor/llama.cpp/src:/tmp/tmpaeyk5c4_/build/vendor/llama.cpp/ggml/src: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/src/libllama.so vendor/llama.cpp/ggml/src/libggml.so && :
505.2 /usr/bin/ld: warning: libcuda.so.1, needed by vendor/llama.cpp/ggml/src/libggml.so, not found (try using -rpath or -rpath-link)
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemCreate'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemAddressReserve'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemUnmap'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemSetAccess'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuDeviceGet'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemAddressFree'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuGetErrorString'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuDeviceGetAttribute'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemMap'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemRelease'
505.2 /usr/bin/ld: vendor/llama.cpp/ggml/src/libggml.so: undefined reference to `cuMemGetAllocationGranularity'
505.2 collect2: error: ld returned 1 exit status
505.2 ninja: build stopped: subcommand failed. Still looking for the libcuda.so.1 file. And why it needs files in the example directory?
Not only I have waited for version 87, but also, I installed the newest version of devel-ubuntu 24.04, upgraded python 3.10 to 3.12, upgraded cudnn8 to 9.3 and installed cuda 12.5.1 from cuda 12.2. Nothing helps.
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
Wheel in version 0.2.86 builds properly, just like 0.2.85 and all other versions before
Current Behavior
Wheel doesn't build and fails
Environment and Context
5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 x86_64 x86_64 GNU/LinuxFailure Information (for bugs)
Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.
Steps to Reproduce
Run:
Failure Logs