Training detection model with custom dataset

[Zoxive]

I am new to CV so please forgive me if this is my error.

While attempting to run detection via:

python train.py configs/mask_rcnn_repvit_m1_1_fpn_1x_coco.py

I noticed these info messages at the top. After much googling around I could not find a definitive answer if this is expected for a pretrain model to contain classifier weights or not.

2023-09-30 00:34:34,425 - mmdet - INFO - Miss []
2023-09-30 00:34:34,425 - mmdet - INFO - Unexpected ['classifier.classifier.bn.weight', 'classifier.classifier.bn.bias', 'classifier.classifier.bn.running_mean', 'classifier.classifier.bn.running_var', 'classifier.classifier.bn.num_batches_tracked', 'classifier.classifier.l.weight', 'classifier.classifier.l.bias', 'classifier.classifier_dist.bn.weight', 'classifier.classifier_dist.bn.bias', 'classifier.classifier_dist.bn.running_mean', 'classifier.classifier_dist.bn.running_var', 'classifier.classifier_dist.bn.num_batches_tracked', 'classifier.classifier_dist.l.weight', 'classifier.classifier_dist.l.bias']
2023-09-30 00:34:34,591 - mmdet - INFO - load checkpoint from local path: pretrain/repvit_m1_1_distill_300e.pth
2023-09-30 00:34:34,631 - mmdet - INFO - Miss []
2023-09-30 00:34:34,631 - mmdet - INFO - Unexpected ['classifier.classifier.bn.weight', 'classifier.classifier.bn.bias', 'classifier.classifier.bn.running_mean', 'classifier.classifier.bn.running_var', 'classifier.classifier.bn.num_batches_tracked', 'classifier.classifier.l.weight', 'classifier.classifier.l.bias', 'classifier.classifier_dist.bn.weight', 'classifier.classifier_dist.bn.bias', 'classifier.classifier_dist.bn.running_mean', 'classifier.classifier_dist.bn.running_var', 'classifier.classifier_dist.bn.num_batches_tracked', 'classifier.classifier_dist.l.weight', 'classifier.classifier_dist.l.bias']

Is it possibly some wires crossed after the recent commit? https://github.com/THU-MIG/RepViT/commit/b749e1fee4f86102068eca70edbf74da1d3792aa#diff-cd9ebb250c84a1c86ee23bcc16dda442f62aaa4e6201916d61b5c191d0eae46b

[jameslahm]

Thanks for your interest! I'd like to confirm that this is expected for a pretrain model to contain classifier weights. Because the classifier is not needed in transferring the pretrain model to the object detection task, these info messages are expected and normal.

[Zoxive]

Thank you for responding. I have much to learn.

What lead me down that area, is ive trained a RepViT model on a custom dataset.

When i try to export it using export_coreml.py it errors that the model key doesnt exist.. so i changed it to use state_dict which does exist. dict_keys(['meta', 'state_dict', 'optimizer'])

Which then gets further along but then it errors actually loading the pytorch checkpoint

Missing key(s) in state_dict: 
"features.0.0.c.weight",
"features.0.0.bn.weight",
"features.0.0.bn.bias",
"features.0.0.bn.running_mean",
"features.0.0.bn.running_var",
"features.0.2.c.weight",
"features.0.2.bn.weight",
"features.0.2.bn.bias", 
....(trimmed)

Unexpected key(s) in state_dict: 
"backbone.features.0.0.c.weight",
"backbone.features.0.0.bn.weight",
"backbone.features.0.0.bn.bias",
"backbone.features.0.0.bn.running_mean",
"backbone.features.0.0.bn.running_var",
"backbone.features.0.0.bn.num_batches_tracked",
"backbone.features.0.2.c.weight",
"backbone.features.0.2.bn.weight",
"backbone.features.0.2.bn.bias",

...(trimmed)

With a keen eye it looks like all the weights are there, but nested in backbone when the model is expecting them at the root.

This leads me to ask a bunch of questions, as this is at the edge of my knowledge.

• Is the detection/repvit.py which is a mmdetection registered RepViT model not compatible with the timm model defined in model/repvit.py?
• Do i need to define a detection specific timm model that is specific to my architecture? (right now i have no custom features etc, but as i learn and play that may change, my config py file is just the same as configs/mask_rcnn_repvit_m1_1_fpn_1x_coco.py while overwriting the coco num_labels and dataset paths)
• Or can i just do some sort of remap and move all the backbone.* weights to the root aka replace 'backbone.', ''
• Or something else because i have no idea what im doing

[jameslahm]

I think the third is the right answer, i.e., replacing 'backbone' with ''.