微调clip_cn_vit-l-14-336 报错

[xxllp]

我尝试换了下sh 里面的resume 加载的模型路径，然后提示模型参数错误： size mismatch for module.visual.transformer.resblocks.0.mlp.c_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for module.visual.transformer.resblocks.0.ln_2.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for module.visual.transformer.resblocks.0.ln_2.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for module.visual.transformer.resblocks.1.attn.in_proj_weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([2304, 768]). size mismatch for module.visual.transformer.resblocks.1.attn.in_proj_bias: copying a param with shape torch.Size([3072]) from checkpoint, the shape in current model is torch.Size([2304]). size mismatch for module.visual.transformer.resblocks.1.attn.out_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for module.visual.transformer.resblocks.1.attn.out_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for module.visual.transformer.resblocks.1.ln_1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for module.visual.transformer.resblocks.1.ln_1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for module.visual.transformer.resblocks.1.mlp.c_fc.weight: copying a param with shape torch.Size([4096, 1024]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for module.visual.transformer.resblocks.1.mlp.c_fc.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([3072]). size mismatch for module.visual.transformer.resblocks.1.mlp.c_proj.weight: copying a param with shape torch.Size([1024, 4096]) from checkpoint, the shape in current model is torch.Size([768, 3072]).

谁知道这个是要改哪里才行~

[diodeshing]

根据上图的模型对应关系，修改如下图微调sh里面的vision_model、text_model。

[xxllp]

看到了~~