FlagOpen / FlagScale

FlagScale is a large model toolkit based on open-sourced projects.
Other
167 stars 42 forks source link

[Experimental] Update to a better heterogenous impl #157

Closed aoyulong closed 3 months ago

aoyulong commented 3 months ago

This pull request is for re-implementing heterogeneous training in a better way. The goals are as follows:

Please note that the code is in a very early stage and there is no guarantee that it will work well.