merge from Nvidia main - Githubissues

bigcode-project / Megatron-LM

Ongoing research training transformer models at scale

Other

371 stars 48 forks source link

Closed RaymondLi0 closed 1 year ago

mayank31398 commented 1 year ago

Hey @RaymondLi0 , thanks a lot for this effort. Any ETA on this?

RaymondLi0 commented 1 year ago

Hopefully early next week :)

RaymondLi0 commented 1 year ago

Some things are broken (I created a branch named before-merge, in case someone wants to do one of these):

loading previous checkpoints that used the distributed optimizer will not work
the checkpoint merging tools will not work on previous checkpoints (may be fixed later with another merge from NVIDIA's repo)