search

alexrozanski / LlamaChat

Chat with your favourite LLaMA models in a native macOS app
https://llamachat.app
MIT License
1.43k stars 53 forks source link

Failed to load model for eachadea/ggml-vicuna-7b-1.1 #15

Open fakechris opened 1 year ago

fakechris commented 1 year ago

After I downloaded eachadea/ggml-vicuna-7b-1.1's ggml-vicuna-7b-1.1-q4_0.bin model from https://huggingface.co/eachadea/ggml-vicuna-7b-1.1/tree/main, I was able to add Chat Source successfully. However, during the conversation, an error "Failed to load model" occurred. I also tried llama.cpp, and I could load the model only after updating to the latest llama.cpp. The llama.cpp from 5 days ago would also fail to load the model. I'm not sure if the ggml model in llama.cpp has been modified in any way.

alexrozanski commented 1 year ago

Hey @fakechris, I know there have been some changes to llama.cpp in the last week, I'm working on updating the bindings so that these are now supported. I haven't tested Vicuna support specifically either, and that's coming

zakkor commented 1 year ago

Vicuna works with the same sort of parameters as plain llama, but requires the "User:" prompt to be used AFAIK