search

clarkmcc / chitchat

A simple LLM chat front-end that makes it easy to find, download, and mess around with models on your local machine.
https://www.clarkmccauley.com/projects/chitchat
MIT License
64 stars 3 forks source link

Prompt templates #11

Closed clarkmcc closed 1 year ago

clarkmcc commented 1 year ago

Right now the backend automatically formats all prompts with

USER: {prompt}\nSYSTEM:

but as I understand it, this needs to be configurable based on the model in order for the model to produce the best responses.

clarkmcc commented 1 year ago

Much better performance on llama 2 chat models when using the proper prompt templates

https://huggingface.co/TheBloke/Llama-2-13B-chat-GPTQ https://huggingface.co/TheBloke/Llama-2-13B-chat-GPTQ/discussions/5

clarkmcc commented 1 year ago

LLaMA 2 Chat

Warmup

[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.
<</SYS>>
Please answer questions about the following document. Answer only the questions, don’t give a lot of information that was not requested.[/INST]

Template

[INST]{{prompt}}[/INST]

Vicuna

Warmup

A chat between a human and an assistant.

Template

USER: {{prompt}}
ASSISTANT: