search

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
9.06k stars 709 forks source link

new format, new finetune code? #64

Open ehartford opened 1 year ago

ehartford commented 1 year ago

Llamax code it knows how to handle alpaca formatted QA data, but I didnt' see anything in there to handle ShareGPT format data,

How do I finetune with the new format? Your finetune guide (https://github.com/nlpxucan/WizardLM/blob/main/WizardLM/README.md#fine-tuning) still references the 70k dataset

chat-guy commented 1 year ago

Your finetune guide (https://github.com/nlpxucan/WizardLM/blob/main/WizardLM/README.md#fine-tuning) still references the 70k dataset

Yep, noticed this too. Perhaps @nlpxucan forgot to update that particular section of the readme during the last commit (2 days ago).

ehartford commented 1 year ago

perhaps this

https://github.com/nlpxucan/WizardLM/blob/94f9c792df4b91589c8c236a566ddc63d4868ec2/WizardLM/src/train_freeform.py#LL50C5-L50C17

ehartford commented 1 year ago

Or maybe they used FastChat rather than Llamax.

ehartford commented 1 year ago

gonna presume we are using FastChat until I hear otherwise.