search

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.site
MIT License
1.06k stars 64 forks source link

Few bugs with template saving #23

Closed ShawnFumo closed 3 months ago

ShawnFumo commented 6 months ago

I started with the Stability 3b template and modified it for use with Zephyr. I changed the prompt and gave it 4096 context and selected Metal (but kept MLock off since it seems fine without it).

After saving that under a new template and saving the chat setting, there's different kinds of issues.

First if you re-open the chat settings, it says Custom again. And if you start typing to change the template name, it changes some of the settings (inference to llama, prompt changes back to default, context to 2048, metal off). So then you have to change them again before re-saving.

Another issue is if I try to switch from Custom on an existing or new chat to the new template I'd made: inference goes to llama, metal turns off, and temperature changes to 512 (this one really threw me off since I didn't notice at first!). But the prompt and context are correct.

Not sure if there are diff issues for diff inferences or if I had other sampling options diff from the defaults, but those are what I ran into so far.

guinmoon commented 6 months ago

Thanks for the feedback. Saving custom templates still needs to be finalized. I will try to fix the errors in new versions.