Size mismatch

[lwgkzl]

size mismatch for final_logits_bias: copying a param with shape torch.Size([1, 54953]) from checkpoint, the shape in current model is torch.Size([1, 54944]). size mismatch for model.shared.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]). size mismatch for model.encoder.embed_tokens.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]). size mismatch for model.decoder.embed_tokens.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]). size mismatch for lm_head.weight: copying a param with shape torch.Size([54953, 512]) from checkpoint, the shape in current model is torch.Size([54944, 512]).

我把从 https://huggingface.co/lsy641/ESC_Blender_noStrategy/tree/main 的模型放到blend-small文件夹下面之后，运行Python BlenderEmotionalSupport.py 文件， 遇到了这个维度不匹配的错误，请问这是怎么回事呢？

[lwgkzl]

我使用的是codes文件夹下的代码

[Sahandfer]

1. 你先要加载原来的facebook/blenderbot-small，然后在tokenizer上加上所有strategy的tokens，再做model.resize_token_embeddings(len(tokenizer))，然后用torch.load_state把这个checkpoint 加载
2. 如果你只需要nostrategy的这个模型可以试一试 https://huggingface.co/chujiezheng/blenderbot_small-90M-ESC

[lsy641]

54953 是原词表大小， 54944是在54953基础上加上8个策略token和一个CLS token，所以应该是要resize一个词表。

[lwgkzl]

谢谢：）

[PHI6kai]

你搞定了吗？是咋操作的呀