Local model not found

[gkapellmann]

Hello,

So I followed the instructions, and I downloaded the model to the PC, so it is locally hosted. But when running tha app, when entering to localhost:8501 I get an error:

Repository Not Found for url: https://huggingface.co/api/models/ggml-llama-2-7b-chat-q4_0.bin/revision/main.

As if it was looking the model somewhere online.

How can I tell it to look for it locally?

[karimababa]

Hello, Same here! Could you please update this repo? Thanks

[ds-muzalevskiy]

I'm not a repo owner, just using the code for my working projects. I have also faced with similar issue and you can solve it by changing line ggml-llama-2-7b-chat-q4_0.bin in app.py to exact HuggingFace repo location of the model: TheBloke/Llama-2-7B-Chat-GGML Then you need to wait until the model will be downloaded from the hub and you can use it. Obviously this is only the example, you can try different other models as well.

Some more documentation: https://pypi.org/project/ctransformers/

[henri-edh]

I agree with ds-muzalevskiy : Here is the code to be modified as the funtion references a different model that is instructed for download in the README.md : return AutoModelForCausalLM.from_pretrained(

ggml-llama-2-7b-chat-q4_0.bin

    'llama-2-7b-chat.ggmlv3.q2_K.bin', 
    model_type='llama',
    temperature=temperature, 
    top_p = top_p)