Can I use python only apex for gpt_pretrain?

bigscience-workshop / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Other

1.3k stars 211 forks source link

Can I use python only apex for gpt_pretrain? #373

Open Luoyang144 opened 1 year ago

Luoyang144 commented 1 year ago

I'm tring to run pretrain_gpt_single_node.sh in this link, and find not amp_C installed, it needs gcc>=6.0 but I can't change gcc version for now. So will it possible to run this script without amp_C?