search

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.site
MIT License
1.06k stars 64 forks source link

Adding support for Mistral #12

Closed rodas-j closed 8 months ago

rodas-j commented 9 months ago

I just want to say I love what you are doing and I'm happy I stumbled upon this repo.

Is it possible to add support for Mistral AI?

https://huggingface.co/mistralai/Mistral-7B-v0.1

guinmoon commented 9 months ago

I have mistral running fine on macOS. Prompt format: [INST] {{prompt}} [/INST] Screenshot 2023-10-08 at 22 15 43

rodas-j commented 9 months ago

I was trying to run Q2 quantized models on xcode simulator and I run into errors. Maybe I'm trying something that's not possible.

Simulator Screenshot - iPhone 15 Pro - 2023-10-08 at 16 52 56

Simulator Screenshot - iPhone 15 Pro - 2023-10-08 at 17 12 48

guinmoon commented 9 months ago

Maybe there is something wrong with the model file? Screenshot 2023-10-09 at 06 39 14 Screenshot 2023-10-09 at 06 37 58

rodas-j commented 8 months ago

I got it to work. You are right. Thanks!