[Hetero] Support heterogeneous training strategy with different `dp_size`

FlagOpen / FlagScale

FlagScale is a large model toolkit based on open-sourced projects.

Other

167 stars 42 forks source link

Closed heavyrain-lzy closed 1 month ago

heavyrain-lzy commented 1 month ago

Support heterogeneous training strategy with different dp_size.