Convert models to GGUF via llama.cpp

josStorer / RWKV-Runner

A RWKV management and startup tool, full automation, only 8MB. And provides an interface compatible with the OpenAI API. RWKV is a large language model that is fully open source and available for commercial use.

https://www.rwkv.com

MIT License

5.32k stars 504 forks source link

Convert models to GGUF via llama.cpp #400

Open bennmann opened 1 month ago

bennmann commented 1 month ago

Please add llama.cpp quant functionality (convert to Q5_K_L, Q2_XS, etc) to Convert precision section: