AssertionError self.model is not None

[ebdavison]

Trying to run this with CPU only and followed the instruction to install and run this on Linux.

Here is what I get:

$ python3 ./app.py
Running on backend llama.cpp.
Model path is empty.
Use default llama.cpp model path: ./models/llama-2-7b-chat.ggmlv3.q4_0.bin
Model exists in ./models/llama-2-7b-chat.ggmlv3.q4_0.bin.
Traceback (most recent call last):
  File "/opt/Data/Personal/NextCloud/Documents/IT/repos/llama2-webui/./app.py", line 325, in <module>
    main()
  File "/opt/Data/Personal/NextCloud/Documents/IT/repos/llama2-webui/./app.py", line 56, in main
    llama2_wrapper = LLAMA2_WRAPPER(
                     ^^^^^^^^^^^^^^^
  File "/opt/Data/Personal/NextCloud/Documents/IT/repos/llama2-webui/llama2_wrapper/model.py", line 99, in __init__
    self.init_model()
  File "/opt/Data/Personal/NextCloud/Documents/IT/repos/llama2-webui/llama2_wrapper/model.py", line 103, in init_model
    self.model = LLAMA2_WRAPPER.create_llama2_model(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Data/Personal/NextCloud/Documents/IT/repos/llama2-webui/llama2_wrapper/model.py", line 125, in create_llama2_model
    model = Llama(
            ^^^^^^
  File "/home/edavison/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 323, in __init__
    assert self.model is not None
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError

My environment:

$ pip3 freeze | grep -E '(llama|bitsand)'
bitsandbytes==0.40.2
llama-cpp-python==0.1.85
llama2-wrapper==0.1.12
$ python3 -V
Python 3.11.5

[jamartinh]

Same here !

[liltom-eth]

Thanks for the issue. Might be some issue from llama.cpp. Will have a look.

[wtryc]

same happening on my macbook m1 here

[wtryc]

same happening on my macbook m1 here

Fixed it by using gguf model

[liltom-eth]

@ebdavison @jamartinh @wtryc hi now the new release llama2-wrapper=0.1.13 will lock on llama-cpp-python = "0.1.77" to support old ggml models. New release will support gguf models.

[liltom-eth]

73 now supporting gguf models.