search

bigscience-workshop / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2
Other
1.31k stars 213 forks source link

add args_deepspeed_gpt.sh #322

Closed xyn1201 closed 2 years ago

xyn1201 commented 2 years ago

添加megatron_deepspeed加zero的运行脚本，其中包括了和libai对齐后的配置参数