search

YiwuZhong / SGG_from_NLS

[ICCV 2021] Official code for "Learning to Generate Scene Graph from Natural Language Supervision"
Other
100 stars 10 forks source link

Problems while loading checkpoints #2

Closed guikunchen closed 2 years ago

guikunchen commented 2 years ago

After downloading Language_CC-COCO_MotifNet on Google Drive, I change the content of last_checkpoint to the path model_final.pth stored.

Then, while running the test.sh, error occurs.

RuntimeError: Error(s) in loading state_dict for GeneralizedRCNN: size mismatch for roi_heads.relation.predictor.out_oobj.2.weight: copying a param with shape torch.Size([4274, 768]) from checkpoint, the shape in current model is torch.Size([149, 768]). size mismatch for roi_heads.relation.predictor.out_oobj.2.bias: copying a param with shape torch.Size([4274]) from checkpoint, the shape in current model is torch.Size([149]). size mismatch for roi_heads.relation.predictor.out_rel.2.weight: copying a param with shape torch.Size([678, 768]) from checkpoint, the shape in current model is torch.Size([66, 768]). size mismatch for roi_heads.relation.predictor.out_rel.2.bias: copying a param with shape torch.Size([678]) from checkpoint, the shape in current model is torch.Size([66]).

The config.yml in Language_CC-COCO_MotifNet does not match the checkpoint or something goes wrong?

Thank you for your time!

YiwuZhong commented 2 years ago

Thanks for your interest. The error seems to be caused by the mismatch between the deep model and the checkpoint file. The test.sh should be good to create the correct model. So I think the problem came from the wrong way of loading the checkpoint file. How about you follow the instruction of Model Evaluation in README, instead of manually changing the content of last_checkpoint?

guikunchen commented 2 years ago

@YiwuZhong Thanks for your reply. I followed the instruction of Model Evaluation. I download all the files within 'Language_CC-COCO_MotifNet' on GoogleDrive and put them to the 'checkpoints/Language_CC-COCO_MotifNet'. Original content of lastcheckpoint is ./checkpoints/motif-sgdet-36/modelfinal.pth, so I manually change it to my path. If not, it will be another FileNotFound error.

YiwuZhong commented 2 years ago

How about trying the following steps:

  1. Download 'Language_CC-COCO_Uniter' on GoogleDrive, since the default test.sh was for Transformer model instead of MotifNet.
  2. Modify last_checkpoint file to be './checkpoints/Language_CC-COCO_Uniter/model_final.pth'. Please make sure this path can be found in your machine. For example, put the downloaded folder 'Language_CC-COCO_Uniter' in the folder of 'checkpoints' of this codebase.
guikunchen commented 2 years ago

Thanks!!! This works. This error occors because of the wrong setting of MODEL.ROI_RELATION_HEAD.PREDICTOR. Default test.sh is for Transformer model. Evaluation of different models should refer to train.sh.