search

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.tech
MIT License
1.38k stars 90 forks source link

How to install and use your own large model？ #97

Closed dingtine closed 4 weeks ago

dingtine commented 4 weeks ago

Is it about quantizing to a GGUF file, saving it to an iOS device, and then adding it in the model management section?

Is there a related tutorial?

guinmoon commented 4 weeks ago

Hi. Tutorial in development. Copy the .gguf file to the models directory and specify this model when creating a new chat.