search

OpenBMB / BMTrain

Efficient Training (including pre-training and fine-tuning) for Big Models
Apache License 2.0
548 stars 74 forks source link

bf16 optimizer #79

Closed ftgreat closed 1 year ago

ftgreat commented 1 year ago

请问 adam和adam_offload 有计划支持 bf16 么？谢谢

Achazwl commented 1 year ago

在 https://github.com/OpenBMB/BMTrain/pull/136 中支持了 不久会合并到 main 分支