It was found that the deepspeed folder exists. Has the deepspeed source code been modified in this project?

CoinCheung / gdGPT

Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.

Apache License 2.0

90 stars 8 forks source link

It was found that the deepspeed folder exists. Has the deepspeed source code been modified in this project? #27

Open daneren opened 8 months ago

CoinCheung commented 8 months ago

Hi,

Yes, I made a bit modification, so that this deepspeed is a bit different from official codebase.

daneren commented 8 months ago

Thanks,

I am looking for a training solution that does not modify deepspeed and can use pp+zero. Can you explain the difference between deepspeed, or which commit can reflect the modification?

CoinCheung commented 8 months ago

Hi,

I am afraid that I have modified it from the very beginning, so currently there is no such commit that depends on non-modified deepspeed.