jquesnelle / yarn

YaRN: Efficient Context Window Extension of Large Language Models
MIT License
1.32k stars 115 forks source link

Finetune Example #4

Closed M-Chris closed 1 year ago

M-Chris commented 1 year ago

Awesome job on this

Do you have any examples of a fine-tune cli / setup to show llama3b 4096 | 6144?

StrangeTcy commented 1 year ago

My questions are of the same sort, but rather about the training/finetuning args and the dataset to use

DumoeDss commented 1 year ago

So do you have any examples of a fine-tune cli / setup to show llama3b?

M-Chris commented 1 year ago

Realized this was still open. Even though the script is pretty standard would have been great to see at the time of the post, a few use-case/examples with ntk parts and other variations found viable for fine-tuning.. closing it, best of luck 👍

This may be of interest to other readers