Crash on sending prompt

[sussyboiiii]

Im in the Beta, macOS Sonoma, M1 Pro 16GB, loading any model crashed even small ones which worked beforehand.

[psugihara]

Hm, i wonder if the models you're using aren't compatible with the latest llama.cpp anymore (llama.cpp periodically breaks compatibility). That's the only substantive change in the latest beta.

[psugihara]

If you have time, can you download a recently quantized model and let me know if it works?

[sussyboiiii]

Hello, I used the newest llama.cpp version and the same model

[sussyboiiii]

Tested two models both crashed in FreeChat and worked in the terminal.

[psugihara]

dang, no good! thanks for testing.

i'm doing a refactor at the moment to simplify a few things and I'll try to figure out what's causing it.

[sussyboiiii]

After updating FreeChat it now doesn't crash anymore.

[sussyboiiii]

Where did the ability to change the system prompt go? Can't find it anymore.

[psugihara]

that's great news! thanks for testing again.

Where did the ability to change the system prompt go? Can't find it anymore.

I still see it in the FreeChat > Settings panel:

[sussyboiiii]

Oh well, I ment the tab to choose the template or whatever it's called where I can choose between Vicuna or e.g. ChatML.

[psugihara]

Yes, I have removed that and now defer to llama.cpp's conversation templating. The prompt formats are now embedded in GGUFs so that inference servers (like the local llama.cpp server we use) can do it correctly themselves. This makes freechat a lot easier to maintain because I don't have to hardcode template formats (and model -> template mappings) myself as new models are released.

[sussyboiiii]

Ah ok nice! Then this seems to be working as intended. Thank you for your work, have a nice day!