edgar971 / open-chat

A self-hosted, offline, ChatGPT-like chatbot with different LLM support. 100% private, with no data leaving your device.
MIT License
64 stars 8 forks source link

Changing LLM to Llama 2 13b #16

Closed memowrecks closed 5 months ago

memowrecks commented 5 months ago

I am attempting to change the default Llama 2 7b to the 13b variant.

I have downloaded the model, and placed it in the models folder: /models/llama-2-13b-chat.ggmlv3.q4_0.bin

I have configured the download URL to: https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/blob/main/llama-2-13b-chat.ggmlv3.q4_0.bin

But now all I get is a spinning wheel in the UI - it doesn't seem to load the LLM

image

image