huawei-noah / Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
3.03k stars 627 forks source link

CeMAT 加载预训练 ckpt 报错 #239

Closed yangb05 closed 1 year ago

yangb05 commented 1 year ago

完全按照 Readme 流程做微调,处理完数据之后,执行脚本:bash CeMAT_plugins/task_NMT_cemat.sh 进行微调。 报错信息: RuntimeError: Error(s) in loading state_dict for BiTransformerModel: size mismatch for encoder.embed_tokens.weight: copying a param with shape torch.Size([64905, 1024]) from checkpoint, the shape in current model is torch.Size([250035, 1024]). size mismatch for decoder.embed_tokens.weight: copying a param with shape torch.Size([64905, 1024]) from checkpoint, the shape in current model is torch.Size([250035, 1024]). size mismatch for decoder.output_projection.weight: copying a param with shape torch.Size([64905, 1024]) from checkpoint, the shape in current model is torch.Size([250035, 1024]). 请问应该如何解决?

yangb05 commented 1 year ago

数据预处理之后生成的二进制文件中,有一个 dict..txt,我把这个当成 dict.txt 了。实际上 dict.txt 应该是 dict.en.txt 或 dict.ro.txt。