Error when installing llama-python-cpp binding - couldn't load for cuda

Expected Behavior

I expect the binding to be installed successfully when I click "install binding" from the binding zoo

Current Behavior

Currently it is outputting an error when I install the binding. " Installed successfully

Binding reinstalled successfully Couldn't load Llamacpp for cuda with tensorcores. Reverting to CPU Couldn't load Llamacpp!!! Binding broken. Try reinstalling it "

Steps to Reproduce

Please provide detailed steps to reproduce the issue.

Install binding llama cpp
Reinstall binding
When selecting the binding the error displays: It looks like you we couldn't load the model. Here is the error message: 'NoneType' object has no attribute 'Llama'
Post issue in github

Possible Solution

If you have any suggestions on how to fix the issue, please describe them here.

Context

---------------------- Installing python_llama_cpp ---------------------- Installing requirements for hardware configuration nvidia-tensorcores Ignoring bitsandbytes: markers 'platform_system != "Windows"' don't match your environment Collecting bitsandbytes==0.41.1 (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 26)) Downloading https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl (152.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.7/152.7 MB 19.8 MB/s eta 0:00:00 Ignoring llama-cpp-python: markers 'platform_system == "Windows" and platform_machine == "x86_64" and python_version == "3.11"' don't match your environment Ignoring llama-cpp-python: markers 'platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.11"' don't match your environment Requirement already satisfied: accelerate==0.25. in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (0.25.0) Requirement already satisfied: colorama in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 2)) (0.4.6) Requirement already satisfied: datasets in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (2.17.0) Requirement already satisfied: einops in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 4)) (0.7.0) Requirement already satisfied: hqq==0.1.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 5)) (0.1.2) Collecting jinja2==3.1.2 (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 6)) Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB) Requirement already satisfied: lm_eval==0.3.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.3.0) Requirement already satisfied: markdown in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 8)) (3.5.2) Requirement already satisfied: numpy==1.24. in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 9)) (1.24.4) Requirement already satisfied: optimum==1.16. in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 10)) (1.16.2) Requirement already satisfied: pandas in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 11)) (2.2.0) Requirement already satisfied: peft==0.7. in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 12)) (0.7.1) Requirement already satisfied: Pillow>=9.5.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 13)) (10.2.0) Requirement already satisfied: pyyaml in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 14)) (6.0.1) Requirement already satisfied: requests in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 15)) (2.31.0) Requirement already satisfied: rich in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 16)) (13.7.0) Requirement already satisfied: safetensors==0.4.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 17)) (0.4.1) Requirement already satisfied: scipy in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 18)) (1.12.0) Requirement already satisfied: sentencepiece in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 19)) (0.1.99) Requirement already satisfied: tensorboard in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (2.16.1) Requirement already satisfied: transformers==4.37. in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 21)) (4.37.2) Requirement already satisfied: tqdm in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from -r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 22)) (4.66.2) Requirement already satisfied: packaging>=20.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from accelerate==0.25.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (23.2) Requirement already satisfied: psutil in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from accelerate==0.25.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (5.9.8) Requirement already satisfied: torch>=1.10.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from accelerate==0.25.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (2.1.2+cu121) Requirement already satisfied: huggingface-hub in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from accelerate==0.25.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (0.20.2) Requirement already satisfied: timm in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from hqq==0.1.2->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 5)) (0.9.12) Requirement already satisfied: termcolor in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from hqq==0.1.2->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 5)) (2.4.0) Requirement already satisfied: MarkupSafe>=2.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from jinja2==3.1.2->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 6)) (2.1.3) Requirement already satisfied: jsonlines in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (4.0.0) Requirement already satisfied: numexpr in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (2.8.8) Requirement already satisfied: openai>=0.6.4 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.9.0) Requirement already satisfied: pybind11>=2.6.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (2.11.1) Requirement already satisfied: pycountry in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (23.12.11) Requirement already satisfied: pytablewriter in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.2.0) Requirement already satisfied: rouge-score>=0.0.4 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.1.2) Requirement already satisfied: sacrebleu==1.5.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.5.0) Requirement already satisfied: scikit-learn>=0.24.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.4.0) Requirement already satisfied: sqlitedict in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (2.1.0) Requirement already satisfied: tqdm-multiprocess in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.0.11) Requirement already satisfied: zstandard in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.19.0) Requirement already satisfied: coloredlogs in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from optimum==1.16.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 10)) (15.0.1) Requirement already satisfied: sympy in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from optimum==1.16.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 10)) (1.12) Requirement already satisfied: filelock in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from transformers==4.37.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 21)) (3.13.1) Requirement already satisfied: regex!=2019.12.17 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from transformers==4.37.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 21)) (2023.12.25) Requirement already satisfied: tokenizers<0.19,>=0.14 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from transformers==4.37.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 21)) (0.15.0) Requirement already satisfied: portalocker in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from sacrebleu==1.5.0->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (2.8.2) Requirement already satisfied: pyarrow>=12.0.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (15.0.0) Requirement already satisfied: pyarrow-hotfix in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (0.6) Requirement already satisfied: dill<0.3.9,>=0.3.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (0.3.7) Requirement already satisfied: xxhash in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (3.4.1) Requirement already satisfied: multiprocess in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (0.70.15) Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (2023.10.0) Requirement already satisfied: aiohttp in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (3.9.1) Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pandas->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 11)) (2.8.2) Requirement already satisfied: pytz>=2020.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pandas->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 11)) (2023.3.post1) Requirement already satisfied: tzdata>=2022.7 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pandas->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 11)) (2023.4) Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from requests->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 15)) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from requests->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 15)) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from requests->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 15)) (1.26.18) Requirement already satisfied: certifi>=2017.4.17 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from requests->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 15)) (2023.11.17) Requirement already satisfied: markdown-it-py>=2.2.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from rich->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 16)) (3.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from rich->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 16)) (2.17.2) Requirement already satisfied: absl-py>=0.4 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (2.1.0) Requirement already satisfied: grpcio>=1.48.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (1.60.0) Requirement already satisfied: protobuf!=4.24.0,>=3.19.6 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (3.20.1) Requirement already satisfied: setuptools>=41.0.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (69.1.0) Requirement already satisfied: six>1.9 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (1.16.0) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (0.7.2) Requirement already satisfied: tf-keras>=2.15.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (2.15.0) Requirement already satisfied: werkzeug>=1.0.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from tensorboard->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 20)) (3.0.1) Requirement already satisfied: attrs>=17.3.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from aiohttp->datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (23.2.0) Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from aiohttp->datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (6.0.4) Requirement already satisfied: yarl<2.0,>=1.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from aiohttp->datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (1.9.4) Requirement already satisfied: frozenlist>=1.1.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from aiohttp->datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (1.4.1) Requirement already satisfied: aiosignal>=1.1.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from aiohttp->datasets->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 3)) (1.3.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from huggingface-hub->accelerate==0.25.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (4.9.0) Requirement already satisfied: mdurl~=0.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from markdown-it-py>=2.2.0->rich->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 16)) (0.1.2) Requirement already satisfied: anyio<5,>=3.5.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (4.2.0) Requirement already satisfied: distro<2,>=1.7.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.8.0) Requirement already satisfied: httpx<1,>=0.23.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.26.0) Requirement already satisfied: pydantic<3,>=1.9.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (2.6.1) Requirement already satisfied: sniffio in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.3.0) Requirement already satisfied: nltk in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from rouge-score>=0.0.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (3.8.1) Requirement already satisfied: joblib>=1.2.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from scikit-learn>=0.24.1->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.3.2) Requirement already satisfied: threadpoolctl>=2.0.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from scikit-learn>=0.24.1->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (3.2.0) Requirement already satisfied: networkx in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from torch>=1.10.0->accelerate==0.25.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 1)) (3.2.1) Requirement already satisfied: humanfriendly>=9.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from coloredlogs->optimum==1.16.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 10)) (10.0) Requirement already satisfied: DataProperty<2,>=1.0.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.0.1) Requirement already satisfied: mbstrdecoder<2,>=1.0.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.1.3) Requirement already satisfied: pathvalidate<4,>=2.3.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (3.2.0) Requirement already satisfied: tabledata<2,>=1.3.1 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.3.3) Requirement already satisfied: tcolorpy<1,>=0.0.5 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.1.4) Requirement already satisfied: typepy<2,>=1.3.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from typepy[datetime]<2,>=1.3.2->pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.3.2) Requirement already satisfied: mpmath>=0.19 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from sympy->optimum==1.16.->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 10)) (1.3.0) Requirement already satisfied: torchvision in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from timm->hqq==0.1.2->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 5)) (0.16.2+cu121) Requirement already satisfied: httpcore==1. in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from httpx<1,>=0.23.0->openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (1.0.2) Requirement already satisfied: h11<0.15,>=0.13 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from httpcore==1.->httpx<1,>=0.23.0->openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.14.0) Requirement already satisfied: pyreadline3 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from humanfriendly>=9.1->coloredlogs->optimum==1.16.*->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 10)) (3.4.1) Requirement already satisfied: chardet<6,>=3.0.4 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from mbstrdecoder<2,>=1.0.0->pytablewriter->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (5.2.0) Requirement already satisfied: annotated-types>=0.4.0 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pydantic<3,>=1.9.0->openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (0.6.0) Requirement already satisfied: pydantic-core==2.16.2 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from pydantic<3,>=1.9.0->openai>=0.6.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (2.16.2) Requirement already satisfied: click in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from nltk->rouge-score>=0.0.4->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (8.1.7) Requirement already satisfied: pywin32>=226 in c:\users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\lib\site-packages (from portalocker->sacrebleu==1.5.0->lm_eval==0.3.0->-r C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo\python_llama_cpp\requirements_nvidia.txt (line 7)) (306) Installing collected packages: jinja2 Attempting uninstall: jinja2 Found existing installation: Jinja2 3.1.3 Uninstalling Jinja2-3.1.3: Successfully uninstalled Jinja2-3.1.3 Successfully installed jinja2-3.1.2 WARNING: Skipping C:\Users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\Lib\site-packages\Jinja2-3.1.3.dist-info due to invalid metadata entry 'name' WARNING: Skipping C:\Users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\Lib\site-packages\Jinja2-3.1.3.dist-info due to invalid metadata entry 'name' WARNING: Skipping C:\Users\ba251\lollms-webui\scripts\windows\installer_files\lollms_env\Lib\site-packages\Jinja2-3.1.3.dist-info due to invalid metadata entry 'name' Installed successfully

Binding reinstalled successfully Couldn't load Llamacpp for cuda with tensorcores. Reverting to CPU Couldn't load Llamacpp!!! Binding broken. Try reinstalling it INFO: ::1:29793 - "POST /reinstall_binding HTTP/1.1" 200 OK

Screenshots

If applicable, add screenshots to help explain the issue. Error:

ParisNeo / lollms-webui