Question about downloading checkpoints

bigscience-workshop / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Other

1.31k stars 213 forks source link

Question about downloading checkpoints #315

Closed mayank31398 closed 2 years ago

mayank31398 commented 2 years ago

Does BigScience also provide the original BLOOM checkpoints (without conversion to Huggingface 🤗). I am working on finetuning BLOOM and I need those checkpoint files.

mayank31398 commented 2 years ago

@stas00 this is blocking my team. Can someone help?

mayank31398 commented 2 years ago

Thanks for the discussion: https://huggingface.co/bigscience/bloom/discussions/54 TLDR: Download DS checkpoints at https://huggingface.co/bigscience/bloom-optimizer-states