Open William-Wildridge opened 1 month ago
I am planning to train Corpus with Llama 3.1 8b instruct and I noticed instruct model requires specific prompt. so I customized the function like this for Corpus training(SFTtraining)
def formatting_func(example):
user_message = example["sentence"]
formatted_prompt = f"<|begin_of_text|><|start_header_id|>user<|end_header_id|>{user_message}<|eom_id|><|start_header_id|>assistant<|end_header_id|>"
return formatted_prompt
I haven't tried it yet with it but will it work? or will it be problematic?
I think this approach goes some way, and I'm curious if it's enough, but <|eot_id|> is not used in this example at all and I think omitting this could have a similar affect on model performance. I don't know if omitting one versus the other is preferrable, but it would be great to have a solution where only
Oh for tool use - yes <|eom_id|> should be used - I'll need to create a new chat template specifically it seems for tool usage
Just wondering if anyone has successfully created a Google Colab notebook regarding finetuning tools for Llama 3.1? Thanks!
Interesting idea - I might make a notebook for tool calling!
With llama 3.1 <|eom_id|> has been introduced to help support multi-turn reasoning.
End of message. A message represents a possible stopping point for execution where the model can inform the executor that a tool call needs to be made. This is used for multi-step interactions between the model and any available tools. This token is emitted by the model when the Environment: ipython instruction is used in the system prompt, or if the model calls for a built-in tool.
Currently get_chat_template is returning function calls with <|eot_id|> at the end of each request.
Docs here: https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/