Open kretes opened 1 year ago
Hi, any news on this? Are the weights ignored on purpose or not?
+1, have same question!
Since the weights are missing in the checkpoint and expected in the model, I am assuming they will be randomly initialized and trained with the downstream task. Any other insights workarounds ?
Instructions To Reproduce the Issue:
tools/lazyconfig_train_net.py --config-file projects/ViTDet/configs/COCO/mask_rcnn_vitdet_h_75ep.py "dataloader.train.total_batch_size=1"
Expected behavior:
The logs above are about loading the ImageNet pretrained on MAE checkpoint into a VitDet. The messages about incompatible shapes & missing weights in the backbone are unexpected and lead me to a belief this is a wrong checkpoint for the model. I think it boils down ito missing weights in patterns:
In case those were were ignored on purpose when exporting checkpoint - I think it would be best to specify the
expected missing weights
(as e.g. the rpn and roi_heads are not expected to be in this checkpoint). If not - maybe good idea to add a print before loading the checkpoint about the expected output, or a comment in configuration file in places like https://github.com/facebookresearch/detectron2/blob/main/projects/ViTDet/configs/COCO/mask_rcnn_vitdet_h_75ep.py#L12Environment:
Paste the output of the following command: