fix adam bf16 load changed to fp16 - Githubissues

OpenBMB / BMTrain

Efficient Training (including pre-training and fine-tuning) for Big Models

Apache License 2.0

548 stars 74 forks source link

fix adam bf16 load changed to fp16 #175

Closed Achazwl closed 10 months ago

Achazwl commented 10 months ago

Pull Request Template

Description

When load from bf16 AdamOffload state_dict, an empty fp16 buffer is created.

Type of Change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update