search

psugihara / FreeChat

llama.cpp based AI chat app for macOS
https://www.freechat.run
MIT License
425 stars 37 forks source link

Improve reliability when switching between models #55

Closed psugihara closed 8 months ago

psugihara commented 8 months ago

Sometimes when I switch between models, the bot becomes unresponsive for a message or two. I think this is from messaging it before the new model has loaded in. We check for the server to be ready by listening to its stdout. I bet there is a more reliable way like pinging the /health endpoint (didn't exist when we started this project).

psugihara commented 8 months ago

too annoying, fixing. I think this regression emerged with the llama.cpp version on the TestFlight.