how do you train the instruc-tuned model?

OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

https://optimalscale.github.io/LMFlow/

Apache License 2.0

8.26k stars 826 forks source link

how do you train the instruc-tuned model? #85

Closed zixiliuUSC closed 1 year ago

zixiliuUSC commented 1 year ago

Do you train the models in model zoo in whole model finetune manner or in LoRA manner? What data do you use in training Hu, Dongshan, Hetian and Altay??

research4pan commented 1 year ago

Thanks for your attention! We train all Hu, Dongshan, Hetian and Altay models in LoRA manner, with alpaca + https://huggingface.co/datasets/BelleGroup/train_0.5M_CN 😄 We are working on training better models with more datasets.