Question about size mismatch in trained models.

[kasuo511]

Thanks for sharing your sharing!

I've tried to modify the SiamRPNRes22 to SiamRPNRes19 in parameters: """ ResNet with 19 layer utilized in CVPR2019 paper. Usage: ResNet(Bottleneck_CI, [2, 4], [True, False], [False, True], 64, [64, 128]) """

and got 20 models in exactly same size, like 610.8 MiB.

Then I tested those models by python ./siamese_tracking/test_siamrpn.py --arch SiamRPNRes19 --resume ./snapshot/checkpoint_e50.pth --dataset VOT2017 --cls_type thinner

and got size mismatch errors like:

RuntimeError: Error(s) in loading state_dict for SiamRPNRes19: size mismatch for connect_model.template_cls.weight: copying a param of torch.Size([1280, 512, 3, 3]) from checkpoint, where the shape is torch.Size([5120, 512, 3, 3]) in current model. size mismatch for connect_model.template_cls.bias: copying a param of torch.Size([1280]) from checkpoint, where the shape is torch.Size([5120]) in current model. size mismatch for connect_model.template_reg.weight: copying a param of torch.Size([5120, 512, 3, 3]) from checkpoint, where the shape is torch.Size([10240, 512, 3, 3]) in current model. size mismatch for connect_model.template_reg.bias: copying a param of torch.Size([5120]) from checkpoint, where the shape is torch.Size([10240]) in current model. size mismatch for connect_model.search_cls.weight: copying a param of torch.Size([256, 512, 3, 3]) from checkpoint, where the shape is torch.Size([512, 512, 3, 3]) in current model. size mismatch for connect_model.search_cls.bias: copying a param of torch.Size([256]) from checkpoint, where the shape is torch.Size([512]) in current model. size mismatch for connect_model.search_reg.weight: copying a param of torch.Size([256, 512, 3, 3]) from checkpoint, where the shape is torch.Size([512, 512, 3, 3]) in current model. size mismatch for connect_model.search_reg.bias: copying a param of torch.Size([256]) from checkpoint, where the shape is torch.Size([512]) in current model.

could you pls to tell me where can I modify and solve the mismatch problem?

[JudasDie]

Thanks for sharing your sharing!

I've tried to modify the SiamRPNRes22 to SiamRPNRes19 in parameters: """ ResNet with 19 layer utilized in CVPR2019 paper. Usage: ResNet(Bottleneck_CI, [2, 4], [True, False], [False, True], 64, [64, 128]) """

and got 20 models in exactly same size, like 610.8 MiB.

Then I tested those models by python ./siamese_tracking/test_siamrpn.py --arch SiamRPNRes19 --resume ./snapshot/checkpoint_e50.pth --dataset VOT2017 --cls_type thinner

and got size mismatch errors like:

RuntimeError: Error(s) in loading state_dict for SiamRPNRes19: size mismatch for connect_model.template_cls.weight: copying a param of torch.Size([1280, 512, 3, 3]) from checkpoint, where the shape is torch.Size([5120, 512, 3, 3]) in current model. size mismatch for connect_model.template_cls.bias: copying a param of torch.Size([1280]) from checkpoint, where the shape is torch.Size([5120]) in current model. size mismatch for connect_model.template_reg.weight: copying a param of torch.Size([5120, 512, 3, 3]) from checkpoint, where the shape is torch.Size([10240, 512, 3, 3]) in current model. size mismatch for connect_model.template_reg.bias: copying a param of torch.Size([5120]) from checkpoint, where the shape is torch.Size([10240]) in current model. size mismatch for connect_model.search_cls.weight: copying a param of torch.Size([256, 512, 3, 3]) from checkpoint, where the shape is torch.Size([512, 512, 3, 3]) in current model. size mismatch for connect_model.search_cls.bias: copying a param of torch.Size([256]) from checkpoint, where the shape is torch.Size([512]) in current model. size mismatch for connect_model.search_reg.weight: copying a param of torch.Size([256, 512, 3, 3]) from checkpoint, where the shape is torch.Size([512, 512, 3, 3]) in current model. size mismatch for connect_model.search_reg.bias: copying a param of torch.Size([256]) from checkpoint, where the shape is torch.Size([512]) in current model.

could you pls to tell me where can I modify and solve the mismatch problem?

Hi, thanks for your interest. It seems that the channels in RPN head are not correct. Could u check it again? BTW, it's suggested to test all epochs instead of the last epoch.

[kasuo511]

there are codes related, pls check these: `class SiamRPNRes19(SiamRPN_): def init(self, kwargs): super(SiamRPNRes19, self).init(kwargs) self.features = ResNet19() inchannels = self.features.feature_size

    if self.cls_type == 'thinner': outchannels = 256
    elif self.cls_type == 'thicker': outchannels = 512
    else: raise ValueError('not implemented loss/cls type')

    self.connect_model = RPN_Up(anchor_nums=self.anchor_nums,
                                inchannels=inchannels,
                                outchannels=outchannels,
                                cls_type = self.cls_type)`

`class RPN_Up(nn.Module): """ For SiamRPN """ def init(self, anchor_nums=5, inchannels=256, outchannels=256, cls_type='thicker'): super(RPN_Up, self).init()

    self.anchor_nums = anchor_nums
    self.inchannels = inchannels
    self.outchannels = outchannels

    if cls_type == 'thinner': self.cls_channel = self.anchor_nums
    elif cls_type == 'thicker': self.cls_channel = self.anchor_nums * 2
    else: raise ValueError('not implemented cls/loss type')

    self.reg_channel = 4 * self.anchor_nums

    self.template_cls = nn.Conv2d(self.inchannels, self.outchannels * self.cls_channel, kernel_size=3)
    self.template_reg = nn.Conv2d(self.inchannels, self.outchannels * self.reg_channel, kernel_size=3)

    self.search_cls = nn.Conv2d(self.inchannels, self.outchannels, kernel_size=3)
    self.search_reg = nn.Conv2d(self.inchannels, self.outchannels, kernel_size=3)
    self.adjust = nn.Conv2d(self.reg_channel, self.reg_channel, kernel_size=1)

def _conv2d_group(self, x, kernel):
    batch = kernel.size()[0]
    pk = kernel.view(-1, x.size()[1], kernel.size()[2], kernel.size()[3])
    px = x.view(1, -1, x.size()[2], x.size()[3])
    po = F.conv2d(px, pk, groups=batch)
    po = po.view(batch, -1, po.size()[2], po.size()[3])
    return po

def forward(self, z_f, x_f):
    cls_kernel = self.template_cls(z_f)
    reg_kernel = self.template_reg(z_f)

    cls_feature = self.search_cls(x_f)
    loc_feature = self.search_reg(x_f)

    _, _, s_cls, _ = cls_kernel.size()
    _, _, s_reg, _ = reg_kernel.size()

    pred_cls = self._conv2d_group(cls_feature, cls_kernel)
    pred_reg = self.adjust(self._conv2d_group(loc_feature, reg_kernel))

    return pred_cls, pred_reg`

[JudasDie]

cls_type

It seems you use different cls_type between training and testing?

[kasuo511]

That's it! That is where my problem lies.It bothered me for weeks. Thank you again for your patient. You may close the issue.

[JudasDie]

That's it! That is where my problem lies.It bothered me for weeks. Thank you again for your patient. You may close the issue.

Glad that I can help you. You can also try this backbone to see whether it brings better results.

[safaaskin]

cls_type

It seems you use different cls_type between training and testing?

thanks alot