problem of loading from_pretrained('princeton-nlp/CoFi-XXX')

[twwwwx]

I just test model = CoFiBertForSequenceClassification.from_pretrained("princeton-nlp/CoFi-MNLI-s95") when I receive an error due to dimension mismatch ` File "/root/token_prune/CoFiPruning-pretrain/test.py", line 16, in model = CoFiBertForSequenceClassification.from_pretrained("princeton-nlp/CoFi-MNLI-s95") File "/opt/conda/lib/python3.7/site-packages/transformers/modeling_utils.py", line 2493, in from_pretrained keep_in_fp32_modules=keep_in_fp32_modules, File "/opt/conda/lib/python3.7/site-packages/transformers/modeling_utils.py", line 2844, in _load_pretrained_model raise RuntimeError(f"Error(s) in loading state_dict for {model.class.name}:\n\t{error_msg}") RuntimeError: Error(s) in loading state_dict for CoFiBertForSequenceClassification: size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30522, 764]) from checkpoint, the shape in current model is torch.Size([30522, 768]). size mismatch for bert.embeddings.position_embeddings.weight: copying a param with shape torch.Size([512, 764]) from checkpoint, the shape in current model is torch.Size([512, 768]).'

[xiamengzhou]

What is the transfomers version you are using?

[xiamengzhou]

Hi, I think it's compatible with transformers==4.17.0 and datasets==1.14.0, please check it out!