search

jllllll / llama-cpp-python-cuBLAS-wheels

Wheels for llama-cpp-python compiled with cuBLAS support
The Unlicense
94 stars 40 forks source link

Can you update all your llama-cpp-python modules to 0.2.29 I'm getting python loading errors in new models. #42

Open NonaSuomy opened 10 months ago

NonaSuomy commented 10 months ago

https://huggingface.co/acon96/Home-3B-v2-GGUF/resolve/main/Home-3B-v2.q8_0.gguf

https://github.com/abetlen/llama-cpp-python/releases/tag/v0.2.29

AMD Vega64 Unbuntu 22

On v1 it loaded fine on v2 you get this error asked the model maintainer they said it's because of the old 0.2.26 version and you need a new version.

Traceback (most recent call last):

File "/home/nonasuomy/code/text-generation-webui/modules/ui_model_menu.py", line 213, in load_model_wrapper

shared.model, shared.tokenizer = load_model(selected_model, loader)

                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nonasuomy/code/text-generation-webui/modules/models.py", line 87, in load_model

output = load_func_map[loader](model_name)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nonasuomy/code/text-generation-webui/modules/models.py", line 250, in llamacpp_loader

model, tokenizer = LlamaCppModel.from_pretrained(model_file)

                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nonasuomy/code/text-generation-webui/modules/llamacpp_model.py", line 101, in from_pretrained

result.model = Llama(**params)

               ^^^^^^^^^^^^^^^
File "/home/nonasuomy/code/text-generation-webui/installer_files/env/lib/python3.11/site-packages/llama_cpp_cuda/llama.py", line 962, in init

self._n_vocab = self.n_vocab()

                ^^^^^^^^^^^^^^
File "/home/nonasuomy/code/text-generation-webui/installer_files/env/lib/python3.11/site-packages/llama_cpp_cuda/llama.py", line 2274, in n_vocab

return self._model.n_vocab()

       ^^^^^^^^^^^^^^^^^^^^^
File "/home/nonasuomy/code/text-generation-webui/installer_files/env/lib/python3.11/site-packages/llama_cpp_cuda/llama.py", line 251, in n_vocab

assert self.model is not None

       ^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Thank you.

Limour-dev commented 9 months ago

Perhaps you could fork this repository, manually trigger the actions yourself, and then use the addresses in releases?

AmineDjeghri commented 9 months ago

@Limour-dev & @NonaSuomy oobabooga is updating the wheels from time to time https://github.com/oobabooga/llama-cpp-python-cuBLAS-wheels/releases/tag/wheels