search

abetlen / llama-cpp-python

Python bindings for llama.cpp
https://llama-cpp-python.readthedocs.io
MIT License
8.04k stars 954 forks source link

Build fail for version from 0.2.80 to 0.2.83 #1616

Open congson1293 opened 3 months ago

congson1293 commented 3 months ago

I installed llama-cpp-python on the system with: CPU AMD EPYC 7542 GPU V100 But it raised the exception shown in the image below: Screenshot 2024-07-23 at 14 12 41

abderrahmane-mhd commented 3 months ago

I had the same problem because CUBLABS is deprecated, you can build 0.2.79 using CUBLABS but for versions >=0.2.8, you should use CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --upgrade --force-reinstall

I think that you have same problem as this issue https://github.com/abetlen/llama-cpp-python/issues/1573#issuecomment-2214088672

congson1293 commented 3 months ago

I had the same problem because CUBLABS is deprecated, you can build 0.2.79 using CUBLABS but for versions >=0.2.8, you should use CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --upgrade --force-reinstall

I think that you have same problem as this issue #1573 (comment)

I tried installing it on the server with RTX3090 card but it still raised the same issue.