Sometimes I need to set a stop token for a model but I'm fine using the default for things like temperature, max tokens, etc. I think we should make it a top level config like chatPromptTemplate
So I could do
{
"name" : "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO",
"description" : "Nous Hermes 2 Mixtral 8x7B DPO is the new flagship Nous Research model trained over the Mixtral 8x7B MoE LLM.",
"websiteUrl" : "https://nousresearch.com/",
"chatPromptTemplate" : "<|im_start|>system\n{{#if @root.preprompt}}{{@root.preprompt}}<|im_end|>\n{{/if}}{{#each messages}}{{#ifUser}}<|im_start|>user\n{{content}}<|im_end|>\n<|im_start|>assistant\n{{/ifUser}}{{#ifAssistant}}{{content}}<|im_end|>\n{{/ifAssistant}}{{/each}}",
"stop": ["<|im_end|>"]
}
Instead of
{
"name" : "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO",
"description" : "Nous Hermes 2 Mixtral 8x7B DPO is the new flagship Nous Research model trained over the Mixtral 8x7B MoE LLM.",
"websiteUrl" : "https://nousresearch.com/",
"chatPromptTemplate" : "<|im_start|>system\n{{#if @root.preprompt}}{{@root.preprompt}}<|im_end|>\n{{/if}}{{#each messages}}{{#ifUser}}<|im_start|>user\n{{content}}<|im_end|>\n<|im_start|>assistant\n{{/ifUser}}{{#ifAssistant}}{{content}}<|im_end|>\n{{/ifAssistant}}{{/each}}",
"parameters": {
"temperature": 0.6,
"top_p": 0.95,
"repetition_penalty": 1.2,
"top_k": 50,
"truncate": 3072,
"max_new_tokens": 1024,
"stop": ["<|im_end|>"]
}
}
Or make all the other params optional, not sure which one is best 🤔
Sometimes I need to set a stop token for a model but I'm fine using the default for things like temperature, max tokens, etc. I think we should make it a top level config like
chatPromptTemplate
So I could do
Instead of
Or make all the other params optional, not sure which one is best 🤔