RuntimeError: Error(s) in loading state_dict for CPTForConditionalGeneration: size mismatch for model.encoder.embeddings.position_ids: copying a param with shape torch.Size([1, 1024]) from checkpoint, the shape in current model is torch.Size([1, 512]). size mismatch for model.encoder.embeddings.position_embeddings.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([512, 1024]). You may consider addingignore_mismatched_sizes=Truein the modelfrom_pretrainedmethod.
RuntimeError: Error(s) in loading state_dict for CPTForConditionalGeneration: size mismatch for model.encoder.embeddings.position_ids: copying a param with shape torch.Size([1, 1024]) from checkpoint, the shape in current model is torch.Size([1, 512]). size mismatch for model.encoder.embeddings.position_embeddings.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([512, 1024]). You may consider adding
ignore_mismatched_sizes=Truein the model
from_pretrainedmethod.
加载cpt-base和cpt-large的时候都报这个错,这是不是config文件的维度写错了导致初始化的模型维度和权重维度不匹配