Open andymartin opened 10 months ago
ok first, have you actually tried building the project with commands suggested in llama-cpp-python readme? because it seems you are trying to build straight to vendor llama.cpp.
Try the following commands on windows (cmd):
git clone --recurse-submodules https://github.com/abetlen/llama-cpp-python.git llama-cpp-python
cd lama-cpp-python
set FORCE_CMAKE=1 && set CMAKE_ARGS=-DLLAMA_CUBLAS=on
python -m pip install -e . --force-reinstall --no-cache-dir
Note: this requires cuda installed and vs studio with cmake
Secondly, what does your script look like? you need to set gpu_layers param to use the gpu.
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
I expect CUDA support to work, or not to be claimed to work.
Current Behavior
CUDA does not work.
Environment and Context
Win11 x64, followed all instructions. Still runs 100% CPU.
$ lscpu
$ uname -a
Failure Information (for bugs)
Your instructions are wrong.
Steps to Reproduce
There is no setup.py.
Try the following:
git clone https://github.com/abetlen/llama-cpp-python
cd llama-cpp-python
rm -rf _skbuild/
# delete any old buildspython setup.py develop
cd ./vendor/llama.cpp
cmake
llama.cpp./main
with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, log an issue with llama.cppFailure Logs
There is no setup.py. Nothing else matters because your instructions don't even match the reality of which files exist.