liltom-eth / llama2-webui

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

MIT License

1.97k stars 202 forks source link

GGML deprecated - support GGUF models? #72

Closed agilebean closed 1 year ago

agilebean commented 1 year ago

Hi, the Huggingface download page for GGML models remarks:

The GGML format has now been superseded by GGUF. As of August 21st 2023, [llama.cpp](https://github.com/ggerganov/llama.cpp) no longer supports GGML models. Third party clients and libraries are expected to still support it for a time, but many may also drop support.

Please use the GGUF models instead.

Does this repo also support GGUF already?

liltom-eth commented 1 year ago

@agilebean thanks for the advice. Will support gguf in the next PR.

liltom-eth commented 1 year ago

73 now supporting gguf models.

agilebean commented 1 year ago

you guys are amazing! thanks a lot!