How to continue fine-tuning training on the basis of your pre training fine-tuning nodes【LaVIN-7B】or 【LaVIN-13B】

luogen1996 / LaVIN

[NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"

502 stars 36 forks source link

Open hangzeli08 opened 1 year ago

hangzeli08 commented 1 year ago

怎么在你们预训练微调节点的基础上接着微调训练？就是想在LaVIN-7B或LaVIN-13B上接着微调训练应该怎么办啊～

luogen1996 commented 10 months ago

这可能需要修改部分训练代码。具体来说，在模型构建完毕之后，训练之前加载我们的adapter权重即可。

zhangjoy2 commented 8 months ago

同问，具体来说，如何操作？