修复模型初始化的Bug

OpenMOSS / CoLLiE

Collaborative Training of Large Language Models in an Efficient Way

https://openlmlab-collie.readthedocs.io

Apache License 2.0

410 stars 58 forks source link

Closed Kausal-Lei closed 1 year ago

Kausal-Lei commented 1 year ago

修复流水线并行/非并行/普通并行从0初始化的Bug 修复流水线并行/非并行/普通并行部分初始化的Bug 在llama/model.py中添加keep字段，用于保存模型需要单独处理初始化的部分，需要单独处理的模块需要有reinitialize_weight函数

KaiLv69 commented 1 year ago

给model写个post_init方法，就不需要给norm层写reinit_weight方法了

KaiLv69 commented 1 year ago

base.py里pipelinemodel加上info，告诉用户暂时没有使用传进来的loss_fn，会在trainer里使用