search

microsoft / DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
https://www.deepspeed.ai/
Apache License 2.0
35.67k stars 4.15k forks source link

where is pretrain_zeropp_gpt.py #3840

Open guoyejun opened 1 year ago

guoyejun commented 1 year ago

hi, pretrain_zeropp_gpt.py is mentioned in zeropp tutorials at https://github.com/microsoft/DeepSpeed/blob/f8551b439ea4a849ecb37d148e3bacbffd64121f/docs/_tutorials/zeropp.md?plain=1#L54 , could you share where to get the file? thanks. I could not find it under https://github.com/microsoft/Megatron-DeepSpeed.

abhilash1910 commented 1 year ago

@HeyangQin could you please let us know regarding pretrain_zeropp_gpt.py ? Thanks

etoilestar commented 1 year ago

@HeyangQin I need pretrain_zeropp_gpt.py, thanks

puppetm4st3r commented 4 months ago

any news on this? thanks

deadfffool commented 3 months ago

I want it, too.