search

bigscience-workshop / xmtf

Crosslingual Generalization through Multitask Finetuning
https://arxiv.org/abs/2211.01786
Apache License 2.0
507 stars 37 forks source link

how to convert model weights(e.g., bigscience/bloomz-560m-optimizer-states) to Hugging Face model.bin file? #16

Closed qazwsx042 closed 1 year ago

qazwsx042 commented 1 year ago

Hi, how to convert model weights(e.g., bigscience/bloomz-560m-optimizer-states) to Hugging Face model.bin file?

Muennighoff commented 1 year ago

You can use this script: https://github.com/huggingface/transformers/blob/main/src/transformers/models/bloom/convert_bloom_original_checkpoint_to_pytorch.py

E.g.

python convert_bloom_original_checkpoint_to_pytorch.py --bloom_checkpoint_path bloomz-560m-optimizer-states --pytorch_dump_folder_path ./ --pretraining_tp 1

qazwsx042 commented 1 year ago

@Muennighoff Thanks!