[FlagScale]Revert parallel_state.py and add new parallel_state.py in folder `flagscale`

FlagOpen / FlagScale

FlagScale is a large model toolkit based on open-sourced projects.

Other

132 stars 40 forks source link

[FlagScale]Revert parallel_state.py and add new parallel_state.py in folder `flagscale` #195

Closed heavyrain-lzy closed 4 weeks ago

heavyrain-lzy commented 1 month ago

Revert parallel_state.py and add new parallel_state.py in folder flagscale, which brings convenience to merge the latest upstream branch of Megatron-LM.