Fix gradient accumulation type in the megatron backend. - Githubissues

openpsi-project / ReaLHF

Super-Efficient RLHF Training of LLMs with Parameter Reallocation

Apache License 2.0

114 stars 4 forks source link

Fix gradient accumulation type in the megatron backend. #33

Closed garrett4wade closed 3 months ago

garrett4wade commented 3 months ago

.