search

simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp
Apache License 2.0
139 stars 20 forks source link

Update llm_llama_cpp.py to support .gguf files. Update README. Closes #10. #16

Closed programmylife closed 1 year ago

programmylife commented 1 year ago

I've updated llm_llama_cpp.py to accept .gguf files instead of .bin files as they are the new standard (per discussion in #10). I updated the README to reflect that change and use .gguf models in place of the .bin files and added an example for the new python model, which may be of particular interest to users.

I removed the instructions for installing via the wheel since that will use the old code which would reject downloads with .gguf. I thought this was a better solution than explaining the difference and potentially having users have an out of date version which won't be able to use new models. I'm happy to discuss alternatives, though.

simonw commented 1 year ago

Thanks for this. I'm going to remove the file extension check entirely and merge this - I don't want to get caught out in the future if they change the extension again.

The README updates are excellent.