Open XunfunLee opened 2 months ago
I checked #1352 , and is there an issus related to windows 11? I just thought it this the problem of my installation steps or my machine. Is there an official explaination plz !!!
Im having the same problem but with Linux, 20,04, using Kaggle Notebook, worked fine until yesterday.
edit: pip install llama-cpp-python==0.2.64
solves the problem.
Im having the same problem but with Linux, 20,04, using Kaggle Notebook, worked fine until yesterday.
edit:
pip install llama-cpp-python==0.2.64
solves the problem.
Still not working, I have been trying 0.2.64, 0.2.60, 0.2.59 for many times and it seems to said:
Creating "ggml_shared.dir\Release\ggml_shared.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
Touching "ggml_shared.dir\Release\ggml_shared.tlog\unsuccessfulbuild".
CustomBuild:
Building Custom Rule C:/Users/Administrator/AppData/Local/Temp/pip-install-_thkprn2/llama-cpp-python_9fa670d7909f4acfb3ac1882363d1df6/vendor/llama.cpp/CMakeLists.txt
The lama.dll is Win32 and we are 64 Bit on Windows 11, if I debug the C++ checker program in Win32 then the lama.dll loads successfully, but for 64 bit nope.
int main() { // Update this path to the actual location of the llama_cpp.dll HINSTANCE hDLL = LoadLibrary(TEXT("C:\my_path\llama-cpp-python\llama_cpp\llama.dll"));
if (hDLL == NULL) {
std::cerr << "ERROR: unable to load DLL" << std::endl;
return 1;
}
std::cout << "DLL loaded successfully" << std::endl;
FreeLibrary(hDLL);
return 0;
}
The lama.dll is Win32 and we are 64 Bit on Windows 11, if I debug the C++ checker program in Win32 then the lama.dll loads successfully, but for 64 bit nope. #include #include
int main() { // Update this path to the actual location of the llama_cpp.dll HINSTANCE hDLL = LoadLibrary(TEXT("C:\my_path\llama-cpp-python\llama_cpp\llama.dll"));
if (hDLL == NULL) { std::cerr << "ERROR: unable to load DLL" << std::endl; return 1; } std::cout << "DLL loaded successfully" << std::endl; FreeLibrary(hDLL); return 0;
} Well I think i can understand why, but I still don't know how to fix this problem, can you give more info or steps plz?
cuda version i'm using v12.4
, windows 10
i think it will also work with windows 11
I have tried this from Windows PowerShell and it works for me
$env:CMAKE_ARGS="-DLLAMA_CUBLAS=on"
$env:CUDACXX="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin\nvcc.exe"
pip install --upgrade --force-reinstall --no-cache-dir llama-cpp-python
I'm having the same issue. I have CUDA installed, nvcc works, and CUDA_PATH is set. Doing: set CMAKE_ARGS=-DLLAMA_CUDA=ON pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose
Don't see any errors in the installation. Yet when I run it I get BLAS = 0:
AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 |
Getting the same result with: set CMAKE_ARGS=-DLLAMA_CUBLAS=ON pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose
cuda version i'm using
v12.4
,windows 10
i think it will also work with windows 11I have tried this from Windows PowerShell and it works for me
$env:CMAKE_ARGS="-DLLAMA_CUBLAS=on" $env:CUDACXX="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin\nvcc.exe" pip install --upgrade --force-reinstall --no-cache-dir llama-cpp-python
It seems to be the problem of windows 11, i made it worked in windows 10 months ago (while the version of llama-cpp-python==0.1.72), but when i turn into the latest version with win11 it doesn't work :(
I'm having the same issue. I have CUDA installed, nvcc works, and CUDA_PATH is set. Doing: set CMAKE_ARGS=-DLLAMA_CUDA=ON pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose
Don't see any errors in the installation. Yet when I run it I get BLAS = 0:
AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 |
Getting the same result with: set CMAKE_ARGS=-DLLAMA_CUBLAS=ON pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose
Yeap, if you make it work plz let me know :) I will keep trying to find the solution as well.
Although it didn't work initially, I was able to download the prebuilt wheel and it works, now I am getting GPU inference. It does seem like there is an issue with my environment in some way.
Issues
I am trying to install the lastest version of llama-cpp-python in my windows 11 with RTX-3090ti(24G). I have successfully installed llama-cpp-python=0.1.87 (can't exactly remember) months ago while using:
But when I want to access the latest version recently by using:
After loading the model, it is still using CPU with
BLAS=0
(or is another params = 1 instead ofBLAS
in new version?).I have been trying the pre-build wheel for CUDA 12.1 (
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
) and it still doesn't work. I add--verbose
to see the output:Environment
python=3.12 C++ compiler: viusal studio 2022 (with necessary C++ modules)
cmake --version
= 3.29.2nvcc -V
= CUDA 12.1 (whilenvidia-smi
cuda version is 12.3, i think it is not related to this issues)I have been download and install VS2022, CUDA toolkit, cmake and anaconda, I am wondering if some steps are missing. Considering my previous experience there is no need to git clone this code and cd into it to build (Though I did that on my mac to build a pth file to bin file months ago).
My system variables are listed below:
Questions
pip install llama-cpp-python
instead of loading model to checkBLAS=1.
pip install llama-cpp-py
?