Open sestinj opened 10 months ago
The llama2 and codellama class of models use a chat template that looks like this:
[INST] <<SYS>> {system_message} <<SYS>> {user_input} [/INST] {response}
But other models use different templates. For example, the Alpaca series of models uses a pattern like this:
### Instruction: {system_message} ### Input: {user_input} ### Response: {response}
To add a prompt template you should:
chat.ts
edit.ts
TemplateType
config_schema.json
autodetectTemplateType
autodetectTemplateFunction
autodetectPromptTemplates
core/llm/index.ts
The llama2 and codellama class of models use a chat template that looks like this:
But other models use different templates. For example, the Alpaca series of models uses a pattern like this:
To add a prompt template you should:
chat.tsedit.ts, following the pattern shown there of starting the response for the LLM.TemplateTypetype, and update the corresponding array inconfig_schema.jsonautodetectTemplateType,autodetectTemplateFunction, andautodetectPromptTemplatesfunctions incore/llm/index.ts.