Open victordonat0 opened 9 months ago
Same issue noticed here on Ubuntu 22.04 with following specs:
Python 3.10.8
GNU Make 4.3 Built for x86_64-pc-linux-gnu
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Facing the same error in installation
I am facing the same error too. I am trying to run it on kaggle.
It used to work earlier if we used:
!CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python
!git clone https://github.com/ggerganov/llama.cpp.git
Here is an example notebook where it worked: https://www.kaggle.com/code/gpreda/fast-test-of-llama-v2-pre-quantized-with-llama-cpp
Same over here! Interested in a way to solve this issue
In the meantime, this worked for me for v0.2.23:
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 \
pip install llama-cpp-python[server]==0.2.23
I tried
CMAKE_ARGS=-DLLAMA_CUBLAS=on FORCE_CMAKE=1 pip install --force-reinstall --no-cache-dir --upgrade llama-cpp-python==0.2.23
and I get
CMake Error at vendor/llama.cpp/CMakeLists.txt:725 (target_link_libraries):
Target "ggml" links to:
CUDA::cublasLt
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Same here can't install a CuBLAS version. It only installs the CPU version so can't use the 2 T4s.
Any help is appreciated.
Same here on Ubuntu 22.04 and CUDA 12.4.99. In my case, trying version==0.2.23 or even 0.2.22 gave the same errors.
Downloading llama_cpp_python-0.2.56.tar.gz (36.9 MB) ....
Building wheels for collected packages: llama-cpp-python Running command Building wheel for llama-cpp-python (pyproject.toml) scikit-build-core 0.8.2 using CMake 3.28.3 (wheel) Configuring CMake... 2024-03-15 03:55:55,466 - scikit_build_core - WARNING - libdir/ldlibrary: /home/jupyter-romp/.conda/envs/llama/lib/libpython3.10.a is not a real file! 2024-03-15 03:55:55,466 - scikit_build_core - WARNING - Can't find a Python library, got libdir=/home/jupyter-romp/.conda/envs/llama/lib, ldlibrary=libpython3.10.a, multiarch=x86_64-linux-gnu, masd=None loading initial cache file /tmp/tmp1zkis1hd/build/CMakeInit.txt -- The C compiler identification is GNU 11.4.0 -- The CXX compiler identification is GNU 11.4.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Git: /usr/bin/git (found version "2.34.1") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Found CUDAToolkit: /home/jupyter-rmurphy/.conda/envs/llama/targets/x86_64-linux/include (found version "12.4.99") -- cuBLAS found -- The CUDA compiler identification is NVIDIA 12.4.99 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /home/jupyter-rmurphy/.conda/envs/llama/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Using CUDA architectures: 52;61;70 -- CUDA host compiler is GNU 11.4.0
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with LLAMA_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- x86 detected CMake Warning (dev) at CMakeLists.txt:21 (install): Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION. This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at CMakeLists.txt:30 (install): Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION. This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done (3.3s) CMake Error at vendor/llama.cpp/CMakeLists.txt:1115 (target_link_libraries): Target "ggml" links to:
CUDA::cublas
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
CMake Error at vendor/llama.cpp/CMakeLists.txt:1122 (target_link_libraries): Target "ggml_shared" links to:
CUDA::cublas
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
CMake Error at vendor/llama.cpp/CMakeLists.txt:1136 (target_link_libraries): Target "llama" links to:
CUDA::cublas
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Testing on a Docker container with the following specifications:
Ubuntu 22.04 Python 3.10.12 Driver Version: 550.54.14 NVIDIA-SMI 550.54.14 CUDA Version: 12.4 First, check if an Cuda architecture is detected using the following commands:
nvidia-smi nvcc --version
Both commands should provide an output. Finally, updating the GPU driver fix my issues.
Used Code for GPU USING:
CUDACXX=/usr/local/cuda-12/bin/nvcc CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=all-major" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade
After a few hours of tinkering, I just ended up removing the whole Conda environment, along with a couple of others that also had CUDA toolkit in them, and then built a new Conda env. from scratch. The install/compile went smoothly after that. I did notice a couple of packages with updated versions, so not sure exactly the fix- version issues or weird cross-environment things.
@p1rate5s can u help me how to set up a new conda env in kaggle ?
@p1rate5s can u help me how to set up a new conda env in kaggle ?
Sorry, I don't use Kaggle. My tinkering is on a bare metal server running Ubuntu. Below are the steps I took to create an env with most tools we would use in our lab, but I certainly cannot recommend them since I am no expert. I have used Python and pipenv before but new to Conda- and I am not sure mixing Conda and pip installs are a really good idea. So, YMMV.
- sudo -E conda create -n llama -c rapidsai -c conda-forge -c nvidia rapids=24.02 python=3.10 cuda-version=12.4 dash streamlit pytorch cupy
- python -m ipykernel install --user --name llama --display-name "llama"
- conda activate llama
- export CMAKE_ARGS="-DLLAMA_CUBLAS=on"
- export FORCE_CMAKE=1
- pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose
- pip install llama-index
- pip install llama-index-readers-file
- pip install llama-index-vector-stores-postgres
- pip install llama-index-embeddings-huggingface
- pip install llama-index-llms-llama-cpp
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
when run
!CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python[server]
should install as expected. but only install on version <=0.2.23
, on0.2.24
or higher not installCurrent Behavior
Environment and Context
Kaggle machine, JUPYTER NOTEBOOK
$ lscpu
$ uname -a