Transfer learning

[doit-ceo]

I'm trying to get transfer learning work, and followed every post I found here. I have 2 cans dataset (pepsi and lipton) and so far I did the following :

First all my images are box of 550px

In config.py did the following changes

COCO_CLASSES = ('lipton', 'pepsi')

COCO_LABEL_MAP = { 1:  1,  2:  2}

Then added my database to same file

cans_dataset = dataset_base.copy({
    'name': 'cans',

    'train_images': '/content/drive/My Drive/yolactx/train/',
    'train_info':   '/content/drive/My Drive/yolactx/train/annotations.json',

    'valid_images': '/content/drive/My Drive/yolactx/test/',
    'valid_info':   '/content/drive/My Drive/yolactx/test/annotations.json',

    'has_gt': True,
    'class_names': ('lipton', 'pepsi')
})

And did the change to the database under yolact_base_config

# Dataset stuff
    'dataset': cans_dataset,
    'num_classes': len(cans_dataset.class_names) + 1,

Also modified the yolact.py to enable retraining

(line 490~)

        #self.load_state_dict(state_dict)
        try:
            self.load_state_dict(state_dict)
        except RuntimeError as e:
            print('Ignoring "' + str(e) + '"')

and (line 630~)

#p = pred_layer(pred_x)
p = pred_layer(pred_x.detach())

I then run the training with

!python ./yolact/train.py --config=yolact_base_config --resume=/content/yolact/weights/yolact_plus_resnet50_54_800000.pth --start_iter=0

I do get some warning (but from my understanding that is normal)

Ignoring "Error(s) in loading state_dict for Yolact:
    Missing key(s) in state_dict: "backbone.layers.2.6.conv1.weight", "backbone.layers.2.6.bn1.weight", "backbone.layers.2.6.bn1.bias", "backbone.layers.2.6.bn1.running_mean", "backbone.layers.2.6.bn1.running_var", "backbone.layers.2.6.conv2.weight", "backbone.layers.2.6.bn2.weight", "backbone.layers.2.6.bn2.bias", "backbone.layers.2.6.bn2.running_mean", "backbone.layers.2.6.bn2.running_var", "backbone.layers.2.6.conv3.weight", "backbone.layers.2.6.bn3.weight", "backbone.layers.2.6.bn3.bias", "backbone.layers.2.6.bn3.running_mean", "backbone.layers.2.6.bn3.running_var", "backbone.layers.2.7.conv1.weight", "backbone.layers.2.7.bn1.weight", "backbone.layers.2.7.bn1.bias", "backbone.layers.2.7.bn1.running_mean", "backbone.layers.2.7.bn1.running_var", "backbone.layers.2.7.conv2.weight", "backbone.layers.2.7.bn2.weight", "backbone.layers.2.7.bn2.bias", "backbone.layers.2.7.bn2.running_mean", "backbone.layers.2.7.bn2.running_var", "backbone.layers.2.7.conv3.weight", "backbone.layers.2.7.bn3.weight", "backbone.layers.2.7.bn3.bias", "backbone.layers.2.7.bn3.running_mean", "backbone.layers.2.7.bn3.running_var", "backbone.layers.2.8.conv1.weight", "backbone.layers.2.8.bn1.weight", "backbone.layers.2.8.bn1.bias", "backbone.layers.2.8.bn1.running_mean", "backbone.layers.2.8.bn1.running_var", "backbone.layers.2.8.conv2.weight", "backbone.layers.2.8.bn2.weight", "backbone.layers.2.8.bn2.bias", "backbone.layers.2.8.bn2.running_mean", "backbone.layers.2.8.bn2.running_var", "backbone.layers.2.8.conv3.weight", "backbone.layers.2.8.bn3.weight", "backbone.layers.2.8.bn3.bias", "backbone.layers.2.8.bn3.running_mean", "backbone.layers.2.8.bn3.running_var", "backbone.layers.2.9.conv1.weight", "backbone.layers.2.9.bn1.weight", "backbone.layers.2.9.bn1.bias", "backbone.layers.2.9.bn1.running_mean", "backbone.layers.2.9.bn1.running_var", "backbone.layers.2.9.conv2.weight", "backbone.layers.2.9.bn2.weight", "backbone.layers.2.9.bn2.bias", "backbone.layers.2.9.bn2.running_mean", "backbone.layers.2.9.bn2.running_var", "backbone.layers.2.9.conv3.weight", "backbone.layers.2.9.bn3.weight", "backbone.layers.2.9.bn3.bias", "backbone.layers.2.9.bn3.running_mean", "backbone.layers.2.9.bn3.running_var", "backbone.layers.2.10.conv1.weight", "backbone.layers.2.10.bn1.weight", "backbone.layers.2.10.bn1.bias", "backbone.layers.2.10.bn1.running_mean", "backbone.layers.2.10.bn1.running_var", "backbone.layers.2.10.conv2.weight", "backbone.layers.2.10.bn2.weight", "backbone.layers.2.10.bn2.bias", "backbone.layers.2.10.bn2.running_mean", "backbone.layers.2.10.bn2.running_var", "backbone.layers.2.10.conv3.weight", "backbone.layers.2.10.bn3.weight", "backbone.layers.2.10.bn3.bias", "backbone.layers.2.10.bn3.running_mean", "backbone.layers.2.10.bn3.running_var", "backbone.layers.2.11.conv1.weight", "backbone.layers.2.11.bn1.weight", "backbone.layers.2.11.bn1.bias", "backbone.layers.2.11.bn1.running_mean", "backbone.layers.2.11.bn1.running_var", "backbone.layers.2.11.conv2.weight", "backbone.layers.2.11.bn2.weight", "backbone.layers.2.11.bn2.bias", "backbone.layers.2.11.bn2.running_mean", "backbone.layers.2.11.bn2.running_var", "backbone.layers.2.11.conv3.weight", "backbone.layers.2.11.bn3.weight", "backbone.layers.2.11.bn3.bias", "backbone.layers.2.11.bn3.running_mean", "backbone.layers.2.11.bn3.running_var", "backbone.layers.2.12.conv1.weight", "backbone.layers.2.12.bn1.weight", "backbone.layers.2.12.bn1.bias", "backbone.layers.2.12.bn1.running_mean", "backbone.layers.2.12.bn1.running_var", "backbone.layers.2.12.conv2.weight", "backbone.layers.2.12.bn2.weight", "backbone.layers.2.12.bn2.bias", "backbone.layers.2.12.bn2.running_mean", "backbone.layers.2.12.bn2.running_var", "backbone.layers.2.12.conv3.weight", "backbone.layers.2.12.bn3.weight", "backbone.layers.2.12.bn3.bias", "backbone.layers.2.12.bn3.running_mean", "backbone.layers.2.12.bn3.running_var", "backbone.layers.2.13.conv1.weight", "backbone.layers.2.13.bn1.weight", "backbone.layers.2.13.bn1.bias", "backbone.layers.2.13.bn1.running_mean", "backbone.layers.2.13.bn1.running_var", "backbone.layers.2.13.conv2.weight", "backbone.layers.2.13.bn2.weight", "backbone.layers.2.13.bn2.bias", "backbone.layers.2.13.bn2.running_mean", "backbone.layers.2.13.bn2.running_var", "backbone.layers.2.13.conv3.weight", "backbone.layers.2.13.bn3.weight", "backbone.layers.2.13.bn3.bias", "backbone.layers.2.13.bn3.running_mean", "backbone.layers.2.13.bn3.running_var", "backbone.layers.2.14.conv1.weight", "backbone.layers.2.14.bn1.weight", "backbone.layers.2.14.bn1.bias", "backbone.layers.2.14.bn1.running_mean", "backbone.layers.2.14.bn1.running_var", "backbone.layers.2.14.conv2.weight", "backbone.layers.2.14.bn2.weight", "backbone.layers.2.14.bn2.bias", "backbone.layers.2.14.bn2.running_mean", "backbone.layers.2.14.bn2.running_var", "backbone.layers.2.14.conv3.weight", "backbone.layers.2.14.bn3.weight", "backbone.layers.2.14.bn3.bias", "backbone.layers.2.14.bn3.running_mean", "backbone.layers.2.14.bn3.running_var", "backbone.layers.2.15.conv1.weight", "backbone.layers.2.15.bn1.weight", "backbone.layers.2.15.bn1.bias", "backbone.layers.2.15.bn1.running_mean", "backbone.layers.2.15.bn1.running_var", "backbone.layers.2.15.conv2.weight", "backbone.layers.2.15.bn2.weight", "backbone.layers.2.15.bn2.bias", "backbone.layers.2.15.bn2.running_mean", "backbone.layers.2.15.bn2.running_var", "backbone.layers.2.15.conv3.weight", "backbone.layers.2.15.bn3.weight", "backbone.layers.2.15.bn3.bias", "backbone.layers.2.15.bn3.running_mean", "backbone.layers.2.15.bn3.running_var", "backbone.layers.2.16.conv1.weight", "backbone.layers.2.16.bn1.weight", "backbone.layers.2.16.bn1.bias", "backbone.layers.2.16.bn1.running_mean", "backbone.layers.2.16.bn1.running_var", "backbone.layers.2.16.conv2.weight", "backbone.layers.2.16.bn2.weight", "backbone.layers.2.16.bn2.bias", "backbone.layers.2.16.bn2.running_mean", "backbone.layers.2.16.bn2.running_var", "backbone.layers.2.16.conv3.weight", "backbone.layers.2.16.bn3.weight", "backbone.layers.2.16.bn3.bias", "backbone.layers.2.16.bn3.running_mean", "backbone.layers.2.16.bn3.running_var", "backbone.layers.2.17.conv1.weight", "backbone.layers.2.17.bn1.weight", "backbone.layers.2.17.bn1.bias", "backbone.layers.2.17.bn1.running_mean", "backbone.layers.2.17.bn1.running_var", "backbone.layers.2.17.conv2.weight", "backbone.layers.2.17.bn2.weight", "backbone.layers.2.17.bn2.bias", "backbone.layers.2.17.bn2.running_mean", "backbone.layers.2.17.bn2.running_var", "backbone.layers.2.17.conv3.weight", "backbone.layers.2.17.bn3.weight", "backbone.layers.2.17.bn3.bias", "backbone.layers.2.17.bn3.running_mean", "backbone.layers.2.17.bn3.running_var", "backbone.layers.2.18.conv1.weight", "backbone.layers.2.18.bn1.weight", "backbone.layers.2.18.bn1.bias", "backbone.layers.2.18.bn1.running_mean", "backbone.layers.2.18.bn1.running_var", "backbone.layers.2.18.conv2.weight", "backbone.layers.2.18.bn2.weight", "backbone.layers.2.18.bn2.bias", "backbone.layers.2.18.bn2.running_mean", "backbone.layers.2.18.bn2.running_var", "backbone.layers.2.18.conv3.weight", "backbone.layers.2.18.bn3.weight", "backbone.layers.2.18.bn3.bias", "backbone.layers.2.18.bn3.running_mean", "backbone.layers.2.18.bn3.running_var", "backbone.layers.2.19.conv1.weight", "backbone.layers.2.19.bn1.weight", "backbone.layers.2.19.bn1.bias", "backbone.layers.2.19.bn1.running_mean", "backbone.layers.2.19.bn1.running_var", "backbone.layers.2.19.conv2.weight", "backbone.layers.2.19.bn2.weight", "backbone.layers.2.19.bn2.bias", "backbone.layers.2.19.bn2.running_mean", "backbone.layers.2.19.bn2.running_var", "backbone.layers.2.19.conv3.weight", "backbone.layers.2.19.bn3.weight", "backbone.layers.2.19.bn3.bias", "backbone.layers.2.19.bn3.running_mean", "backbone.layers.2.19.bn3.running_var", "backbone.layers.2.20.conv1.weight", "backbone.layers.2.20.bn1.weight", "backbone.layers.2.20.bn1.bias", "backbone.layers.2.20.bn1.running_mean", "backbone.layers.2.20.bn1.running_var", "backbone.layers.2.20.conv2.weight", "backbone.layers.2.20.bn2.weight", "backbone.layers.2.20.bn2.bias", "backbone.layers.2.20.bn2.running_mean", "backbone.layers.2.20.bn2.running_var", "backbone.layers.2.20.conv3.weight", "backbone.layers.2.20.bn3.weight", "backbone.layers.2.20.bn3.bias", "backbone.layers.2.20.bn3.running_mean", "backbone.layers.2.20.bn3.running_var", "backbone.layers.2.21.conv1.weight", "backbone.layers.2.21.bn1.weight", "backbone.layers.2.21.bn1.bias", "backbone.layers.2.21.bn1.running_mean", "backbone.layers.2.21.bn1.running_var", "backbone.layers.2.21.conv2.weight", "backbone.layers.2.21.bn2.weight", "backbone.layers.2.21.bn2.bias", "backbone.layers.2.21.bn2.running_mean", "backbone.layers.2.21.bn2.running_var", "backbone.layers.2.21.conv3.weight", "backbone.layers.2.21.bn3.weight", "backbone.layers.2.21.bn3.bias", "backbone.layers.2.21.bn3.running_mean", "backbone.layers.2.21.bn3.running_var", "backbone.layers.2.22.conv1.weight", "backbone.layers.2.22.bn1.weight", "backbone.layers.2.22.bn1.bias", "backbone.layers.2.22.bn1.running_mean", "backbone.layers.2.22.bn1.running_var", "backbone.layers.2.22.conv2.weight", "backbone.layers.2.22.bn2.weight", "backbone.layers.2.22.bn2.bias", "backbone.layers.2.22.bn2.running_mean", "backbone.layers.2.22.bn2.running_var", "backbone.layers.2.22.conv3.weight", "backbone.layers.2.22.bn3.weight", "backbone.layers.2.22.bn3.bias", "backbone.layers.2.22.bn3.running_mean", "backbone.layers.2.22.bn3.running_var". 
    Unexpected key(s) in state_dict: "maskiou_net.maskiou_net.0.weight", "maskiou_net.maskiou_net.0.bias", "maskiou_net.maskiou_net.2.weight", "maskiou_net.maskiou_net.2.bias", "maskiou_net.maskiou_net.4.weight", "maskiou_net.maskiou_net.4.bias", "maskiou_net.maskiou_net.6.weight", "maskiou_net.maskiou_net.6.bias", "maskiou_net.maskiou_net.8.weight", "maskiou_net.maskiou_net.8.bias", "maskiou_net.maskiou_net.10.weight", "maskiou_net.maskiou_net.10.bias", "backbone.layers.1.0.conv2.bias", "backbone.layers.1.0.conv2.conv_offset_mask.weight", "backbone.layers.1.0.conv2.conv_offset_mask.bias", "backbone.layers.1.1.conv2.bias", "backbone.layers.1.1.conv2.conv_offset_mask.weight", "backbone.layers.1.1.conv2.conv_offset_mask.bias", "backbone.layers.1.2.conv2.bias", "backbone.layers.1.2.conv2.conv_offset_mask.weight", "backbone.layers.1.2.conv2.conv_offset_mask.bias", "backbone.layers.1.3.conv2.bias", "backbone.layers.1.3.conv2.conv_offset_mask.weight", "backbone.layers.1.3.conv2.conv_offset_mask.bias", "backbone.layers.2.0.conv2.bias", "backbone.layers.2.0.conv2.conv_offset_mask.weight", "backbone.layers.2.0.conv2.conv_offset_mask.bias", "backbone.layers.2.1.conv2.bias", "backbone.layers.2.1.conv2.conv_offset_mask.weight", "backbone.layers.2.1.conv2.conv_offset_mask.bias", "backbone.layers.2.2.conv2.bias", "backbone.layers.2.2.conv2.conv_offset_mask.weight", "backbone.layers.2.2.conv2.conv_offset_mask.bias", "backbone.layers.2.3.conv2.bias", "backbone.layers.2.3.conv2.conv_offset_mask.weight", "backbone.layers.2.3.conv2.conv_offset_mask.bias", "backbone.layers.2.4.conv2.bias", "backbone.layers.2.4.conv2.conv_offset_mask.weight", "backbone.layers.2.4.conv2.conv_offset_mask.bias", "backbone.layers.2.5.conv2.bias", "backbone.layers.2.5.conv2.conv_offset_mask.weight", "backbone.layers.2.5.conv2.conv_offset_mask.bias", "backbone.layers.3.0.conv2.bias", "backbone.layers.3.0.conv2.conv_offset_mask.weight", "backbone.layers.3.0.conv2.conv_offset_mask.bias", "backbone.layers.3.1.conv2.bias", "backbone.layers.3.1.conv2.conv_offset_mask.weight", "backbone.layers.3.1.conv2.conv_offset_mask.bias", "backbone.layers.3.2.conv2.bias", "backbone.layers.3.2.conv2.conv_offset_mask.weight", "backbone.layers.3.2.conv2.conv_offset_mask.bias". 
    size mismatch for prediction_layers.0.bbox_layer.weight: copying a param with shape torch.Size([36, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([12, 256, 3, 3]).
    size mismatch for prediction_layers.0.bbox_layer.bias: copying a param with shape torch.Size([36]) from checkpoint, the shape in current model is torch.Size([12]).
    size mismatch for prediction_layers.0.conf_layer.weight: copying a param with shape torch.Size([729, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([9, 256, 3, 3]).
    size mismatch for prediction_layers.0.conf_layer.bias: copying a param with shape torch.Size([729]) from checkpoint, the shape in current model is torch.Size([9]).
    size mismatch for prediction_layers.0.mask_layer.weight: copying a param with shape torch.Size([288, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 256, 3, 3]).
    size mismatch for prediction_layers.0.mask_layer.bias: copying a param with shape torch.Size([288]) from checkpoint, the shape in current model is torch.Size([96]).
    size mismatch for semantic_seg_conv.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([2, 256, 1, 1]).
    size mismatch for semantic_seg_conv.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([2])."
Begin training!

Finally interference

!python ./yolact/eval.py --trained_model=./yolact/weights/yolact_plus_resnet50_54_800000.pth --config=yolact_base_config --score_threshold=0.15 --top_k=15 --images=test_images:output_images

Also same warnings as training

Loading model... Ignoring "Error(s) in loading state_dict for Yolact:
    Missing key(s) in state_dict: "backbone.layers.2.6.conv1.weight", "backbone.layers.2.6.bn1.weight", "backbone.layers.2.6.bn1.bias", "backbone.layers.2.6.bn1.running_mean", "backbone.layers.2.6.bn1.running_var", "backbone.layers.2.6.conv2.weight", "backbone.layers.2.6.bn2.weight", "backbone.layers.2.6.bn2.bias", "backbone.layers.2.6.bn2.running_mean", "backbone.layers.2.6.bn2.running_var", "backbone.layers.2.6.conv3.weight", "backbone.layers.2.6.bn3.weight", "backbone.layers.2.6.bn3.bias", "backbone.layers.2.6.bn3.running_mean", "backbone.layers.2.6.bn3.running_var", "backbone.layers.2.7.conv1.weight", "backbone.layers.2.7.bn1.weight", "backbone.layers.2.7.bn1.bias", "backbone.layers.2.7.bn1.running_mean", "backbone.layers.2.7.bn1.running_var", "backbone.layers.2.7.conv2.weight", "backbone.layers.2.7.bn2.weight", "backbone.layers.2.7.bn2.bias", "backbone.layers.2.7.bn2.running_mean", "backbone.layers.2.7.bn2.running_var", "backbone.layers.2.7.conv3.weight", "backbone.layers.2.7.bn3.weight", "backbone.layers.2.7.bn3.bias", "backbone.layers.2.7.bn3.running_mean", "backbone.layers.2.7.bn3.running_var", "backbone.layers.2.8.conv1.weight", "backbone.layers.2.8.bn1.weight", "backbone.layers.2.8.bn1.bias", "backbone.layers.2.8.bn1.running_mean", "backbone.layers.2.8.bn1.running_var", "backbone.layers.2.8.conv2.weight", "backbone.layers.2.8.bn2.weight", "backbone.layers.2.8.bn2.bias", "backbone.layers.2.8.bn2.running_mean", "backbone.layers.2.8.bn2.running_var", "backbone.layers.2.8.conv3.weight", "backbone.layers.2.8.bn3.weight", "backbone.layers.2.8.bn3.bias", "backbone.layers.2.8.bn3.running_mean", "backbone.layers.2.8.bn3.running_var", "backbone.layers.2.9.conv1.weight", "backbone.layers.2.9.bn1.weight", "backbone.layers.2.9.bn1.bias", "backbone.layers.2.9.bn1.running_mean", "backbone.layers.2.9.bn1.running_var", "backbone.layers.2.9.conv2.weight", "backbone.layers.2.9.bn2.weight", "backbone.layers.2.9.bn2.bias", "backbone.layers.2.9.bn2.running_mean", "backbone.layers.2.9.bn2.running_var", "backbone.layers.2.9.conv3.weight", "backbone.layers.2.9.bn3.weight", "backbone.layers.2.9.bn3.bias", "backbone.layers.2.9.bn3.running_mean", "backbone.layers.2.9.bn3.running_var", "backbone.layers.2.10.conv1.weight", "backbone.layers.2.10.bn1.weight", "backbone.layers.2.10.bn1.bias", "backbone.layers.2.10.bn1.running_mean", "backbone.layers.2.10.bn1.running_var", "backbone.layers.2.10.conv2.weight", "backbone.layers.2.10.bn2.weight", "backbone.layers.2.10.bn2.bias", "backbone.layers.2.10.bn2.running_mean", "backbone.layers.2.10.bn2.running_var", "backbone.layers.2.10.conv3.weight", "backbone.layers.2.10.bn3.weight", "backbone.layers.2.10.bn3.bias", "backbone.layers.2.10.bn3.running_mean", "backbone.layers.2.10.bn3.running_var", "backbone.layers.2.11.conv1.weight", "backbone.layers.2.11.bn1.weight", "backbone.layers.2.11.bn1.bias", "backbone.layers.2.11.bn1.running_mean", "backbone.layers.2.11.bn1.running_var", "backbone.layers.2.11.conv2.weight", "backbone.layers.2.11.bn2.weight", "backbone.layers.2.11.bn2.bias", "backbone.layers.2.11.bn2.running_mean", "backbone.layers.2.11.bn2.running_var", "backbone.layers.2.11.conv3.weight", "backbone.layers.2.11.bn3.weight", "backbone.layers.2.11.bn3.bias", "backbone.layers.2.11.bn3.running_mean", "backbone.layers.2.11.bn3.running_var", "backbone.layers.2.12.conv1.weight", "backbone.layers.2.12.bn1.weight", "backbone.layers.2.12.bn1.bias", "backbone.layers.2.12.bn1.running_mean", "backbone.layers.2.12.bn1.running_var", "backbone.layers.2.12.conv2.weight", "backbone.layers.2.12.bn2.weight", "backbone.layers.2.12.bn2.bias", "backbone.layers.2.12.bn2.running_mean", "backbone.layers.2.12.bn2.running_var", "backbone.layers.2.12.conv3.weight", "backbone.layers.2.12.bn3.weight", "backbone.layers.2.12.bn3.bias", "backbone.layers.2.12.bn3.running_mean", "backbone.layers.2.12.bn3.running_var", "backbone.layers.2.13.conv1.weight", "backbone.layers.2.13.bn1.weight", "backbone.layers.2.13.bn1.bias", "backbone.layers.2.13.bn1.running_mean", "backbone.layers.2.13.bn1.running_var", "backbone.layers.2.13.conv2.weight", "backbone.layers.2.13.bn2.weight", "backbone.layers.2.13.bn2.bias", "backbone.layers.2.13.bn2.running_mean", "backbone.layers.2.13.bn2.running_var", "backbone.layers.2.13.conv3.weight", "backbone.layers.2.13.bn3.weight", "backbone.layers.2.13.bn3.bias", "backbone.layers.2.13.bn3.running_mean", "backbone.layers.2.13.bn3.running_var", "backbone.layers.2.14.conv1.weight", "backbone.layers.2.14.bn1.weight", "backbone.layers.2.14.bn1.bias", "backbone.layers.2.14.bn1.running_mean", "backbone.layers.2.14.bn1.running_var", "backbone.layers.2.14.conv2.weight", "backbone.layers.2.14.bn2.weight", "backbone.layers.2.14.bn2.bias", "backbone.layers.2.14.bn2.running_mean", "backbone.layers.2.14.bn2.running_var", "backbone.layers.2.14.conv3.weight", "backbone.layers.2.14.bn3.weight", "backbone.layers.2.14.bn3.bias", "backbone.layers.2.14.bn3.running_mean", "backbone.layers.2.14.bn3.running_var", "backbone.layers.2.15.conv1.weight", "backbone.layers.2.15.bn1.weight", "backbone.layers.2.15.bn1.bias", "backbone.layers.2.15.bn1.running_mean", "backbone.layers.2.15.bn1.running_var", "backbone.layers.2.15.conv2.weight", "backbone.layers.2.15.bn2.weight", "backbone.layers.2.15.bn2.bias", "backbone.layers.2.15.bn2.running_mean", "backbone.layers.2.15.bn2.running_var", "backbone.layers.2.15.conv3.weight", "backbone.layers.2.15.bn3.weight", "backbone.layers.2.15.bn3.bias", "backbone.layers.2.15.bn3.running_mean", "backbone.layers.2.15.bn3.running_var", "backbone.layers.2.16.conv1.weight", "backbone.layers.2.16.bn1.weight", "backbone.layers.2.16.bn1.bias", "backbone.layers.2.16.bn1.running_mean", "backbone.layers.2.16.bn1.running_var", "backbone.layers.2.16.conv2.weight", "backbone.layers.2.16.bn2.weight", "backbone.layers.2.16.bn2.bias", "backbone.layers.2.16.bn2.running_mean", "backbone.layers.2.16.bn2.running_var", "backbone.layers.2.16.conv3.weight", "backbone.layers.2.16.bn3.weight", "backbone.layers.2.16.bn3.bias", "backbone.layers.2.16.bn3.running_mean", "backbone.layers.2.16.bn3.running_var", "backbone.layers.2.17.conv1.weight", "backbone.layers.2.17.bn1.weight", "backbone.layers.2.17.bn1.bias", "backbone.layers.2.17.bn1.running_mean", "backbone.layers.2.17.bn1.running_var", "backbone.layers.2.17.conv2.weight", "backbone.layers.2.17.bn2.weight", "backbone.layers.2.17.bn2.bias", "backbone.layers.2.17.bn2.running_mean", "backbone.layers.2.17.bn2.running_var", "backbone.layers.2.17.conv3.weight", "backbone.layers.2.17.bn3.weight", "backbone.layers.2.17.bn3.bias", "backbone.layers.2.17.bn3.running_mean", "backbone.layers.2.17.bn3.running_var", "backbone.layers.2.18.conv1.weight", "backbone.layers.2.18.bn1.weight", "backbone.layers.2.18.bn1.bias", "backbone.layers.2.18.bn1.running_mean", "backbone.layers.2.18.bn1.running_var", "backbone.layers.2.18.conv2.weight", "backbone.layers.2.18.bn2.weight", "backbone.layers.2.18.bn2.bias", "backbone.layers.2.18.bn2.running_mean", "backbone.layers.2.18.bn2.running_var", "backbone.layers.2.18.conv3.weight", "backbone.layers.2.18.bn3.weight", "backbone.layers.2.18.bn3.bias", "backbone.layers.2.18.bn3.running_mean", "backbone.layers.2.18.bn3.running_var", "backbone.layers.2.19.conv1.weight", "backbone.layers.2.19.bn1.weight", "backbone.layers.2.19.bn1.bias", "backbone.layers.2.19.bn1.running_mean", "backbone.layers.2.19.bn1.running_var", "backbone.layers.2.19.conv2.weight", "backbone.layers.2.19.bn2.weight", "backbone.layers.2.19.bn2.bias", "backbone.layers.2.19.bn2.running_mean", "backbone.layers.2.19.bn2.running_var", "backbone.layers.2.19.conv3.weight", "backbone.layers.2.19.bn3.weight", "backbone.layers.2.19.bn3.bias", "backbone.layers.2.19.bn3.running_mean", "backbone.layers.2.19.bn3.running_var", "backbone.layers.2.20.conv1.weight", "backbone.layers.2.20.bn1.weight", "backbone.layers.2.20.bn1.bias", "backbone.layers.2.20.bn1.running_mean", "backbone.layers.2.20.bn1.running_var", "backbone.layers.2.20.conv2.weight", "backbone.layers.2.20.bn2.weight", "backbone.layers.2.20.bn2.bias", "backbone.layers.2.20.bn2.running_mean", "backbone.layers.2.20.bn2.running_var", "backbone.layers.2.20.conv3.weight", "backbone.layers.2.20.bn3.weight", "backbone.layers.2.20.bn3.bias", "backbone.layers.2.20.bn3.running_mean", "backbone.layers.2.20.bn3.running_var", "backbone.layers.2.21.conv1.weight", "backbone.layers.2.21.bn1.weight", "backbone.layers.2.21.bn1.bias", "backbone.layers.2.21.bn1.running_mean", "backbone.layers.2.21.bn1.running_var", "backbone.layers.2.21.conv2.weight", "backbone.layers.2.21.bn2.weight", "backbone.layers.2.21.bn2.bias", "backbone.layers.2.21.bn2.running_mean", "backbone.layers.2.21.bn2.running_var", "backbone.layers.2.21.conv3.weight", "backbone.layers.2.21.bn3.weight", "backbone.layers.2.21.bn3.bias", "backbone.layers.2.21.bn3.running_mean", "backbone.layers.2.21.bn3.running_var", "backbone.layers.2.22.conv1.weight", "backbone.layers.2.22.bn1.weight", "backbone.layers.2.22.bn1.bias", "backbone.layers.2.22.bn1.running_mean", "backbone.layers.2.22.bn1.running_var", "backbone.layers.2.22.conv2.weight", "backbone.layers.2.22.bn2.weight", "backbone.layers.2.22.bn2.bias", "backbone.layers.2.22.bn2.running_mean", "backbone.layers.2.22.bn2.running_var", "backbone.layers.2.22.conv3.weight", "backbone.layers.2.22.bn3.weight", "backbone.layers.2.22.bn3.bias", "backbone.layers.2.22.bn3.running_mean", "backbone.layers.2.22.bn3.running_var". 
    Unexpected key(s) in state_dict: "maskiou_net.maskiou_net.0.weight", "maskiou_net.maskiou_net.0.bias", "maskiou_net.maskiou_net.2.weight", "maskiou_net.maskiou_net.2.bias", "maskiou_net.maskiou_net.4.weight", "maskiou_net.maskiou_net.4.bias", "maskiou_net.maskiou_net.6.weight", "maskiou_net.maskiou_net.6.bias", "maskiou_net.maskiou_net.8.weight", "maskiou_net.maskiou_net.8.bias", "maskiou_net.maskiou_net.10.weight", "maskiou_net.maskiou_net.10.bias", "backbone.layers.1.0.conv2.bias", "backbone.layers.1.0.conv2.conv_offset_mask.weight", "backbone.layers.1.0.conv2.conv_offset_mask.bias", "backbone.layers.1.1.conv2.bias", "backbone.layers.1.1.conv2.conv_offset_mask.weight", "backbone.layers.1.1.conv2.conv_offset_mask.bias", "backbone.layers.1.2.conv2.bias", "backbone.layers.1.2.conv2.conv_offset_mask.weight", "backbone.layers.1.2.conv2.conv_offset_mask.bias", "backbone.layers.1.3.conv2.bias", "backbone.layers.1.3.conv2.conv_offset_mask.weight", "backbone.layers.1.3.conv2.conv_offset_mask.bias", "backbone.layers.2.0.conv2.bias", "backbone.layers.2.0.conv2.conv_offset_mask.weight", "backbone.layers.2.0.conv2.conv_offset_mask.bias", "backbone.layers.2.1.conv2.bias", "backbone.layers.2.1.conv2.conv_offset_mask.weight", "backbone.layers.2.1.conv2.conv_offset_mask.bias", "backbone.layers.2.2.conv2.bias", "backbone.layers.2.2.conv2.conv_offset_mask.weight", "backbone.layers.2.2.conv2.conv_offset_mask.bias", "backbone.layers.2.3.conv2.bias", "backbone.layers.2.3.conv2.conv_offset_mask.weight", "backbone.layers.2.3.conv2.conv_offset_mask.bias", "backbone.layers.2.4.conv2.bias", "backbone.layers.2.4.conv2.conv_offset_mask.weight", "backbone.layers.2.4.conv2.conv_offset_mask.bias", "backbone.layers.2.5.conv2.bias", "backbone.layers.2.5.conv2.conv_offset_mask.weight", "backbone.layers.2.5.conv2.conv_offset_mask.bias", "backbone.layers.3.0.conv2.bias", "backbone.layers.3.0.conv2.conv_offset_mask.weight", "backbone.layers.3.0.conv2.conv_offset_mask.bias", "backbone.layers.3.1.conv2.bias", "backbone.layers.3.1.conv2.conv_offset_mask.weight", "backbone.layers.3.1.conv2.conv_offset_mask.bias", "backbone.layers.3.2.conv2.bias", "backbone.layers.3.2.conv2.conv_offset_mask.weight", "backbone.layers.3.2.conv2.conv_offset_mask.bias". 
    size mismatch for prediction_layers.0.bbox_layer.weight: copying a param with shape torch.Size([36, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([12, 256, 3, 3]).
    size mismatch for prediction_layers.0.bbox_layer.bias: copying a param with shape torch.Size([36]) from checkpoint, the shape in current model is torch.Size([12]).
    size mismatch for prediction_layers.0.conf_layer.weight: copying a param with shape torch.Size([729, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([9, 256, 3, 3]).
    size mismatch for prediction_layers.0.conf_layer.bias: copying a param with shape torch.Size([729]) from checkpoint, the shape in current model is torch.Size([9]).
    size mismatch for prediction_layers.0.mask_layer.weight: copying a param with shape torch.Size([288, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 256, 3, 3]).
    size mismatch for prediction_layers.0.mask_layer.bias: copying a param with shape torch.Size([288]) from checkpoint, the shape in current model is torch.Size([96]).
    size mismatch for semantic_seg_conv.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([2, 256, 1, 1]).
    size mismatch for semantic_seg_conv.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([2])."
 Done.

Now I'm getting really bad results and I think I made something really wrong !

[doit-ceo]

OK , I partially solved this, I found out that the new weights are stored in a different folder !

So now I run the interference with

!python ./yolact/eval.py --trained_model=./weights/yolact_base_1999_10000.pth --config=yolact_base_config --score_threshold=0.15 --top_k=15 --images=test_images:output_images

And much better results , but as you can see, still something wrong ?!

[ic]

Just a couple comments:

• You are using Yolact++, which requires extra settings and dependencies. All good on that side?
• Transfer learning works best when the new task is a subset of the original task. It seems you are not in this situation, and should expect low performance without significant retraining. The backbone model you are using learned the original COCO classes. By redefining them, your first new class gets mixed with the first COCO class (person), at first and until enough training overrides the original learning. Please note label names are just names, the underlying class ID is where the confusion happens.
• Here I have no error on loading the backbone state dict. I think that is more than a warning. However I do not use Yolact++, just the plain Yolact.

[edwardnguyen1705]

Hi @doit-ceo , !python ./yolact/train.py --config=yolact_base_config --resume=/content/yolact/weights/yolact_plus_resnet50_54_800000.pth --start_iter=0

Your config file and weight file are not consistent.