Error in tracking

[Hatem-Jr]

I chose the data.yaml file like you said but got this error:

RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Yolact: Missing key(s) in state_dict: "backbone.layers.0.0.conv1.weight", "backbone.layers.0.0.bn1.weight", "backbone.layers.0.0.bn1.bias", "backbone.layers.0.0.bn1.running_mean", "backbone.layers.0.0.bn1.running_var", "backbone.layers.0.0.conv2.weight", "backbone.layers.0.0.bn2.weight", "backbone.layers.0.0.bn2.bias", "backbone.layers.0.0.bn2.running_mean", "backbone.layers.0.0.bn2.running_var", "backbone.layers.0.0.conv3.weight", "backbone.layers.0.0.bn3.weight", "backbone.layers.0.0.bn3.bias", "backbone.layers.0.0.bn3.running_mean", "backbone.layers.0.0.bn3.running_var", "backbone.layers.0.0.downsample.0.weight", "backbone.layers.0.0.downsample.1.weight", "backbone.layers.0.0.downsample.1.bias", "backbone.layers.0.0.downsample.1.running_mean", "backbone.layers.0.0.downsample.1.running_var", "backbone.layers.0.1.conv1.weight", "backbone.layers.0.1.bn1.weight", "backbone.layers.0.1.bn1.bias", "backbone.layers.0.1.bn1.running_mean", "backbone.layers.0.1.bn1.running_var", "backbone.layers.0.1.conv2.weight", "backbone.layers.0.1.bn2.weight", "backbone.layers.0.1.bn2.bias", "backbone.layers.0.1.bn2.running_mean", "backbone.layers.0.1.bn2.running_var", "backbone.layers.0.1.conv3.weight", "backbone.layers.0.1.bn3.weight", "backbone.layers.0.1.bn3.bias", "backbone.layers.0.1.bn3.running_mean", "backbone.layers.0.1.bn3.running_var", "backbone.layers.0.2.conv1.weight", "backbone.layers.0.2.bn1.weight", "backbone.layers.0.2.bn1.bias", "backbone.layers.0.2.bn1.running_mean", "backbone.layers.0.2.bn1.running_var", "backbone.layers.0.2.conv2.weight", "backbone.layers.0.2.bn2.weight", "backbone.layers.0.2.bn2.bias", "backbone.layers.0.2.bn2.running_mean", "backbone.layers.0.2.bn2.running_var", "backbone.layers.0.2.conv3.weight", "backbone.layers.0.2.bn3.weight", "backbone.layers.0.2.bn3.bias", "backbone.layers.0.2.bn3.running_mean", "backbone.layers.0.2.bn3.running_var", "backbone.layers.1.0.conv1.weight", "backbone.layers.1.0.bn1.weight", "backbone.layers.1.0.bn1.bias", "backbone.layers.1.0.bn1.running_mean", "backbone.layers.1.0.bn1.running_var", "backbone.layers.1.0.conv2.weight", "backbone.layers.1.0.bn2.weight", "backbone.layers.1.0.bn2.bias", "backbone.layers.1.0.bn2.running_mean", "backbone.layers.1.0.bn2.running_var", "backbone.layers.1.0.conv3.weight", "backbone.layers.1.0.bn3.weight", "backbone.layers.1.0.bn3.bias", "backbone.layers.1.0.bn3.running_mean", "backbone.layers.1.0.bn3.running_var", "backbone.layers.1.0.downsample.0.weight", "backbone.layers.1.0.downsample.1.weight", "backbone.layers.1.0.downsample.1.bias", "backbone.layers.1.0.downsample.1.running_mean", "backbone.layers.1.0.downsample.1.running_var", "backbone.layers.1.1.conv1.weight", "backbone.layers.1.1.bn1.weight", "backbone.layers.1.1.bn1.bias", "backbone.layers.1.1.bn1.running_mean", "backbone.layers.1.1.bn1.running_var", "backbone.layers.1.1.conv2.weight", "backbone.layers.1.1.bn2.weight", "backbone.layers.1.1.bn2.bias", "backbone.layers.1.1.bn2.running_mean", "backbone.layers.1.1.bn2.running_var", "backbone.layers.1.1.conv3.weight", "backbone.layers.1.1.bn3.weight", "backbone.layers.1.1.bn3.bias", "backbone.layers.1.1.bn3.running_mean", "backbone.layers.1.1.bn3.running_var", "backbone.layers.1.2.conv1.weight", "backbone.layers.1.2.bn1.weight", "backbone.layers.1.2.bn1.bias", "backbone.layers.1.2.bn1.running_mean", "backbone.layers.1.2.bn1.running_var", "backbone.layers.1.2.conv2.weight", "backbone.layers.1.2.bn2.weight", "backbone.layers.1.2.bn2.bias", "backbone.layers.1.2.bn2.running_mean", "backbone.layers.1.2.bn2.running_var", "backbone.layers.1.2.conv3.weight", "backbone.layers.1.2.bn3.weight", "backbone.layers.1.2.bn3.bias", "backbone.layers.1.2.bn3.running_mean", "backbone.layers.1.2.bn3.running_var", "backbone.layers.1.3.conv1.weight", "backbone.layers.1.3.bn1.weight", "backbone.layers.1.3.bn1.bias", "backbone.layers.1.3.bn1.running_mean", "backbone.layers.1.3.bn1.running_var", "backbone.layers.1.3.conv2.weight", "backbone.layers.1.3.bn2.weight", "backbone.layers.1.3.bn2.bias", "backbone.layers.1.3.bn2.running_mean", "backbone.layers.1.3.bn2.running_var", "backbone.layers.1.3.conv3.weight", "backbone.layers.1.3.bn3.weight", "backbone.layers.1.3.bn3.bias", "backbone.layers.1.3.bn3.running_mean", "backbone.layers.1.3.bn3.running_var", "backbone.layers.2.0.conv1.weight", "backbone.layers.2.0.bn1.weight", "backbone.layers.2.0.bn1.bias", "backbone.layers.2.0.bn1.running_mean", "backbone.layers.2.0.bn1.running_var", "backbone.layers.2.0.conv2.weight", "backbone.layers.2.0.bn2.weight", "backbone.layers.2.0.bn2.bias", "backbone.layers.2.0.bn2.running_mean", "backbone.layers.2.0.bn2.running_var", "backbone.layers.2.0.conv3.weight", "backbone.layers.2.0.bn3.weight", "backbone.layers.2.0.bn3.bias", "backbone.layers.2.0.bn3.running_mean", "backbone.layers.2.0.bn3.running_var", "backbone.layers.2.0.downsample.0.weight", "backbone.layers.2.0.downsample.1.weight", "backbone.layers.2.0.downsample.1.bias", "backbone.layers.2.0.downsample.1.running_mean", "backbone.layers.2.0.downsample.1.running_var", "backbone.layers.2.1.conv1.weight", "backbone.layers.2.1.bn1.weight", "backbone.layers.2.1.bn1.bias", "backbone.layers.2.1.bn1.running_mean", "backbone.layers.2.1.bn1.running_var", "backbone.layers.2.1.conv2.weight", "backbone.layers.2.1.bn2.weight", "backbone.layers.2.1.bn2.bias", "backbone.layers.2.1.bn2.running_mean", "backbone.layers.2.1.bn2.running_var", "backbone.layers.2.1.conv3.weight", "backbone.layers.2.1.bn3.weight", "backbone.layers.2.1.bn3.bias", "backbone.layers.2.1.bn3.running_mean", "backbone.layers.2.1.bn3.running_var", "backbone.layers.2.2.conv1.weight", "backbone.layers.2.2.bn1.weight", "backbone.layers.2.2.bn1.bias", "backbone.layers.2.2.bn1.running_mean", "backbone.layers.2.2.bn1.running_var", "backbone.layers.2.2.conv2.weight", "backbone.layers.2.2.bn2.weight", "backbone.layers.2.2.bn2.bias", "backbone.layers.2.2.bn2.running_mean", "backbone.layers.2.2.bn2.running_var", "backbone.layers.2.2.conv3.weight", "backbone.layers.2.2.bn3.weight", "backbone.layers.2.2.bn3.bias", "backbone.layers.2.2.bn3.running_mean", "backbone.layers.2.2.bn3.running_var", "backbone.layers.2.3.conv1.weight", "backbone.layers.2.3.bn1.weight", "backbone.layers.2.3.bn1.bias", "backbone.layers.2.3.bn1.running_mean", "backbone.layers.2.3.bn1.running_var", "backbone.layers.2.3.conv2.weight", "backbone.layers.2.3.bn2.weight", "backbone.layers.2.3.bn2.bias", "backbone.layers.2.3.bn2.running_mean", "backbone.layers.2.3.bn2.running_var", "backbone.layers.2.3.conv3.weight", "backbone.layers.2.3.bn3.weight", "backbone.layers.2.3.bn3.bias", "backbone.layers.2.3.bn3.running_mean", "backbone.layers.2.3.bn3.running_var", "backbone.layers.2.4.conv1.weight", "backbone.layers.2.4.bn1.weight", "backbone.layers.2.4.bn1.bias", "backbone.layers.2.4.bn1.running_mean", "backbone.layers.2.4.bn1.running_var", "backbone.layers.2.4.conv2.weight", "backbone.layers.2.4.bn2.weight", "backbone.layers.2.4.bn2.bias", "backbone.layers.2.4.bn2.running_mean", "backbone.layers.2.4.bn2.running_var", "backbone.layers.2.4.conv3.weight", "backbone.layers.2.4.bn3.weight", "backbone.layers.2.4.bn3.bias", "backbone.layers.2.4.bn3.running_mean", "backbone.layers.2.4.bn3.running_var", "backbone.layers.2.5.conv1.weight", "backbone.layers.2.5.bn1.weight", "backbone.layers.2.5.bn1.bias", "backbone.layers.2.5.bn1.running_mean", "backbone.layers.2.5.bn1.running_var", "backbone.layers.2.5.conv2.weight", "backbone.layers.2.5.bn2.weight", "backbone.layers.2.5.bn2.bias", "backbone.layers.2.5.bn2.running_mean", "backbone.layers.2.5.bn2.running_var", "backbone.layers.2.5.conv3.weight", "backbone.layers.2.5.bn3.weight", "backbone.layers.2.5.bn3.bias", "backbone.layers.2.5.bn3.running_mean", "backbone.layers.2.5.bn3.running_var", "backbone.layers.3.0.conv1.weight", "backbone.layers.3.0.bn1.weight", "backbone.layers.3.0.bn1.bias", "backbone.layers.3.0.bn1.running_mean", "backbone.layers.3.0.bn1.running_var", "backbone.layers.3.0.conv2.weight", "backbone.layers.3.0.bn2.weight", "backbone.layers.3.0.bn2.bias", "backbone.layers.3.0.bn2.running_mean", "backbone.layers.3.0.bn2.running_var", "backbone.layers.3.0.conv3.weight", "backbone.layers.3.0.bn3.weight", "backbone.layers.3.0.bn3.bias", "backbone.layers.3.0.bn3.running_mean", "backbone.layers.3.0.bn3.running_var", "backbone.layers.3.0.downsample.0.weight", "backbone.layers.3.0.downsample.1.weight", "backbone.layers.3.0.downsample.1.bias", "backbone.layers.3.0.downsample.1.running_mean", "backbone.layers.3.0.downsample.1.running_var", "backbone.layers.3.1.conv1.weight", "backbone.layers.3.1.bn1.weight", "backbone.layers.3.1.bn1.bias", "backbone.layers.3.1.bn1.running_mean", "backbone.layers.3.1.bn1.running_var", "backbone.layers.3.1.conv2.weight", "backbone.layers.3.1.bn2.weight", "backbone.layers.3.1.bn2.bias", "backbone.layers.3.1.bn2.running_mean", "backbone.layers.3.1.bn2.running_var", "backbone.layers.3.1.conv3.weight", "backbone.layers.3.1.bn3.weight", "backbone.layers.3.1.bn3.bias", "backbone.layers.3.1.bn3.running_mean", "backbone.layers.3.1.bn3.running_var", "backbone.layers.3.2.conv1.weight", "backbone.layers.3.2.bn1.weight", "backbone.layers.3.2.bn1.bias", "backbone.layers.3.2.bn1.running_mean", "backbone.layers.3.2.bn1.running_var", "backbone.layers.3.2.conv2.weight", "backbone.layers.3.2.bn2.weight", "backbone.layers.3.2.bn2.bias", "backbone.layers.3.2.bn2.running_mean", "backbone.layers.3.2.bn2.running_var", "backbone.layers.3.2.conv3.weight", "backbone.layers.3.2.bn3.weight", "backbone.layers.3.2.bn3.bias", "backbone.layers.3.2.bn3.running_mean", "backbone.layers.3.2.bn3.running_var", "backbone.conv1.weight", "backbone.bn1.weight", "backbone.bn1.bias", "backbone.bn1.running_mean", "backbone.bn1.running_var", "proto_net.0.weight", "proto_net.0.bias", "proto_net.2.weight", "proto_net.2.bias", "proto_net.4.weight", "proto_net.4.bias", "proto_net.8.weight", "proto_net.8.bias", "proto_net.10.weight", "proto_net.10.bias", "fpn.lat_layers.0.weight", "fpn.lat_layers.0.bias", "fpn.lat_layers.1.weight", "fpn.lat_layers.1.bias", "fpn.lat_layers.2.weight", "fpn.lat_layers.2.bias", "fpn.pred_layers.0.weight", "fpn.pred_layers.0.bias", "fpn.pred_layers.1.weight", "fpn.pred_layers.1.bias", "fpn.pred_layers.2.weight", "fpn.pred_layers.2.bias", "fpn.downsample_layers.0.weight", "fpn.downsample_layers.0.bias", "fpn.downsample_layers.1.weight", "fpn.downsample_layers.1.bias", "prediction_layers.0.upfeature.0.weight", "prediction_layers.0.upfeature.0.bias", "prediction_layers.0.bbox_layer.weight", "prediction_layers.0.bbox_layer.bias", "prediction_layers.0.conf_layer.weight", "prediction_layers.0.conf_layer.bias", "prediction_layers.0.mask_layer.weight", "prediction_layers.0.mask_layer.bias", "semantic_seg_conv.weight", "semantic_seg_conv.bias". Unexpected key(s) in state_dict: "model", "optimizer", "scheduler", "iteration".

Is it something with the trained model or do I have to change something else ? (the trained model stopped at iteration 999 due to connection issues) and I made sure to edit the data.yaml file so no other problems could occur. I'm also using the same coco dataset that you made when I sent my video to you and have not changed anything when training the model on Detectron

[healthonrails]

It seems that the saved YOLACT model is not valid. You can verify it with a pretrained YOLACT model on default COCO dataset.

[Hatem-Jr]

It seems that the saved YOLACT model is not valid. You can verify it with a pretrained YOLACT model on default COCO dataset.

Thing is it worked in the collab notebook of Detectron in random selected frames in this section: "Inference & evaluation using the trained model" unless you're referencing/suggesting that I do something else ?

[healthonrails]

The Detectron2 Colab will produce the tracking results with the trained Mask-RCNN model. And you don't need to use the Track animals dialog in the user interface.

[Hatem-Jr]

so....the interface is for making custom datasets in coco format only by labels and polygons ? and the collab notebook is to get results from said trained model/dataset ?

ps:- don't worry about closing the issue I'll close it myself :) also if this works you'll be cited in my bachelor thesis :D