search

FlagOpen / FlagScale

FlagScale is a large model toolkit based on open-sourced projects.
Other
132 stars 40 forks source link

[BugFix] Repair train_mixtral_8x7b.yaml #184

Closed shenzhu1993 closed 1 month ago

shenzhu1993 commented 1 month ago

This PR, based on the Megatron-LM repository scripts and the Mixtral-8x7B paper, fixes some bugs that were present in the train_mixtral_8x7b.yaml file and modifies some unnecessary parameters.