Open arthurwolf opened 1 year ago
My recommendation would be to try Cuda 11.8, I have had problems with other installations using Cuda 12 in the past working with LLMs. With Cuda 11.7 and 11.8 I have had no issues. I currently got everything installed with Cuda 11.8 also on Ubuntu using python 3.10.
@arthurwolf You can try building using the following, it worked for me.
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
@arthurwolf You can try building using the following, it worked for me.
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
Thanks it finally worked for me in WSL2
@arthurwolf You can try building using the following, it worked for me.
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
Thanks, it worked
Worked for me with small corrections. My CUDA Version: 12.2 . I use poetry env manager
CUDACXX=/usr/local/cuda-12.0/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 poetry run pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
@arthurwolf You can try building using the following, it worked for me.
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
This works for me too! thank you very much!
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
thanks it works out in ubuntu22.4
@arthurwolf
I have Cuda 12.0 on Ubuntu 22.04 and it works perfectly with your patch. Thanks
@arthurwolf You can try building using the following, it worked for me.
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=native" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
Thanks, the CUDACXX
environmental variable was all I needed.
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
Expected it to build.
Current Behavior
Fails at this point:
Environment and Context
Ubuntu 22.04, Intel CPU, 64GB Ram and 3060 GPU with latest nvidia drivers (535.86.10) and cuda ( 12.2 ) installed via apt.
Steps to Reproduce