FlagOpen / FlagScale

FlagScale is a large model toolkit based on open-sourced projects.
Other
132 stars 40 forks source link

从checkpoint中加载模型进行增量预训练 #200

Closed echo-valor closed 3 weeks ago

echo-valor commented 3 weeks ago

如何从llama3-8b模型进行3D并行划分好的文件进行模型加载训练,目前直接用megatron划分好的checkpoint无法读入进去。 image