search

OpenMOSS / CoLLiE

Collaborative Training of Large Language Models in an Efficient Way
https://openlmlab-collie.readthedocs.io
Apache License 2.0
405 stars 58 forks source link

fix(model.py): align_preciion_in_norm_layer #182

Open gyt1145028706 opened 3 months ago

gyt1145028706 commented 3 months ago

在 Norm Layer 中添加 set_norm_precision_to_float32 函数，在 model 初始化后调用，将 Norm Layer 的权重转成 float32