search

simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp
Apache License 2.0
139 stars 20 forks source link

Fix gguf model registration with no models dir #29

Open georgebashi opened 10 months ago

georgebashi commented 10 months ago

Howdy, I was following the steps in your recent blog post to get set up on a new machine, and I failed at the last step with:

> llm -m gguf -o path mixtral-8x7b-v0.1.Q4_K_M.gguf '[INST] Write a Python function that downloads a file from a URL[/INST]'
Error: 'gguf' is not a known model

Here's a quick PR to fix it 😊 It looks like LlamaGGUF should be registered in all cases, and the existence check for models.json is only relevant for the model-registration logic.

Thank you for such an awesome tool, glad to be able to contribute in a small way!

blech commented 4 months ago

Thank you for this- it also fixed the issue for me (macOS 12.7.4, Python 3.12.4 via Homebrew, llm 0.14)