Update openbuddy prompt template

[thiner]

Is your feature request related to a problem? Please describe. Openbuddy has upgraded the chat template, some of their models are trained to follow the template strictly. Please consider update the template or provide method to overwrite the existing one. The new template: https://github.com/OpenBuddy/OpenBuddy#prompt-format

Describe the solution you'd like Update the template. The new template: https://github.com/OpenBuddy/OpenBuddy#prompt-format

Describe alternatives you've considered Provide a method to overwrite the existing one.

Additional context Affected models Openbuddy Mixtral 7Bx8 V16.x, you may verify with the model. If we don't use the new template, model will mostly tend to ignore system prompt and user instructions.

[abetlen]

@thiner sure thing, is there a new version or something we can use to distinguish the new template from the old one?

[thiner]

I checked with the author of OpenBuddy, he said that the llama-cpp-python currently using template is different from latest one, and redirected me to openbuddy github page to check the latest template. Below is the latest template from the page:

You are a helpful, respectful and honest INTP-T AI Assistant named Buddy. You are talking to a human User.
Always answer as helpfully and logically as possible, while being safe. Your answers should not include any harmful, political, religious, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
You can speak fluently in many languages, for example: English, Chinese.
You cannot access the internet, but you have vast knowledge, cutoff: 2021-09.
You are trained by OpenBuddy team, (https://openbuddy.ai, https://github.com/OpenBuddy/OpenBuddy), you are based on LLaMA and Falcon transformers model, not related to GPT or OpenAI.

User: {History input}
Assistant: {History output}
User: {Input}
Assistant:

As per my understanding, the instruction before User:{History input} is optional. But the template must ensure: there should not be any spaces or line breaks after the last "Assistant:" line.

That's all I know. I tried to build llama-cpp-python from source, but failed unfortunately. I can help to test the new chat template if can provide a built-out image.

[thiner]

I went through relevant code in llama_chat_format.py, I think the problem is easy to resolve. Just change system_message = _system_message to system_message = _get_system_message(messages) or _system_message.

[neowisard]

Just use these files for all openbuddy models(customize in yaml) openbuddy.zip

[thiner]

@neowisard I guess you extracted the tmpl files from LocalAI. That should work for LocalAI, but things just different for llama-cpp-python. The llama-cpp-python doesn't support customize chat template yet, so we have to update code instead. anyway, thanks for your help.

[abetlen]

@thiner should be up-to-date now in main and included in the next release.

[thiner]

@abetlen I tested the latest version, seems it still having problem to follow user instructions. I checked the latest code of llama_chat_format.py, seems the way how to combine built-in system message with user input remains the same, user instruction provided as system prompt will be dropped off at run time, thus it causes problem to follow user instruction. Please consider modify the line as below: system_message = _get_system_message(messages) or _system_message or maybe merge the PR: #913