Bump llama.cpp, -ngl 99, tokens/second fix

psugihara / FreeChat

llama.cpp based AI chat app for macOS

https://www.freechat.run

MIT License

438 stars 41 forks source link

Closed psugihara closed 6 months ago

psugihara commented 6 months ago

see #70

On my mbp I see ~24 tokens/sec with a llama 8B_5_K_M and GPU and ~13 without GPU on.

vercel[bot] commented 6 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
free-chat	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 27, 2024 4:29pm