Missing key(s) in state_dict: "contrastive_align_projection_image.weight", "contrastive_align_projection_image.bias", "contrastive_align_projection_text.weight", "contrastive_align_projection_text.bias".

[MorningStarOvO]

Hello, Thanks for open sourcing! I try to run evaluation on GQA but failed.

python main.py  --dataset_config configs/gqa.json --ema --eval --do_qa --split_qa_heads --resume https://zenodo.org/record/4721981/files/gqa_resnet101_checkpoint.pth

the following error occurred:

Traceback (most recent call last):
  File "main.py", line 649, in <module>
    main(args)
  File "main.py", line 465, in main
    model_without_ddp.load_state_dict(checkpoint["model"])
  File "/home/data/anaconda3/envs/qxy_mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1223, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MDETR:
        Missing key(s) in state_dict: "contrastive_align_projection_image.weight", "contrastive_align_projection_image.bias", "contrastive_align_projection_text.weight", "contrastive_align_projection_text.bias".

How to deal with it?

[ashkamath]

Hi! for the final finetuning on GQA balanced we do not use the contrastive align loss. That's why those keys aren't in the state dict. So just pass --no_contrastive_align_loss along with the other command line args and it should work. I'll update it in the README too. Thanks!

[MorningStarOvO]

Thank you for your comment! The problem is solved.