search

mbzuai-nlp / LaMini-LM

LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
806 stars 49 forks source link

Training Code #2

Open vgoklani opened 1 year ago

vgoklani commented 1 year ago

Are you planning to release the training and model files?

Thanks!

macabdul9 commented 1 year ago

Hi @vgoklani,

All trained models are available here: https://huggingface.co/MBZUAI. For code, currently, we are a little occupied with training and evaluating more models but one can use Standford's Alpaca to train models on LaMini Instructions as we are following the same. For more details please look into our paper: https://arxiv.org/abs/2304.14402

Thanks again.

haiduo commented 4 months ago

Hi @vgoklani,

All trained models are available here: https://huggingface.co/MBZUAI. For code, currently, we are a little occupied with training and evaluating more models but one can use Standford's Alpaca to train models on LaMini Instructions as we are following the same. For more details please look into our paper: https://arxiv.org/abs/2304.14402

Thanks again.

Hello, could you please provide the code for distilling the student model from the teacher model? How is sequence-level distillation done here?