search

NVIDIA / ContrastiveLosses4VRD

Implementation for the CVPR2019 paper "Graphical Contrastive Losses for Scene Graph Generation"
Other
200 stars 41 forks source link

2 FC layers or just 1 for `prd_cls_feats`? #6

Closed juliuswang0728 closed 5 years ago

juliuswang0728 commented 5 years ago

Hi!

I was trying to load trained_models/vg_VGG16/model_step62722.pth and evaluated it on VG. I got errors complaining the incompatible size:

['While copying the parameter named "RelDN.prd_cls_feats.0.bias", whose dimensions in the model are torch.Size([6144]) and whose dimensions in the checkpoint are torch.Size([4096]).',
 'While copying the parameter named "RelDN.prd_cls_feats.0.weight", whose dimensions in the model are torch.Size([6144, 12288]) and whose dimensions in the checkpoint are torch.Size([4096, 12288]).']

Hence, I did some digging and changed the following part in ./lib/modeling_rel/reldn_heads.py because the pretrained weights seems to suggest that there is only one FC layer projecting 12288 to 4096 neurons, whereas in Fig. 3 of the paper it says 2 FC layers are used to project the concatenated features back to 4096. Now I'm able to run the inference and evaluation, just haven't gotten the results. Just wondering is there some mismatch between the provided trained model (in particular, trained_models/vg_VGG16/model_step62722.pth) and the model defined in the codes?

self.prd_cls_feats = nn.Sequential(
            nn.Linear(dim_in, dim_in // 2),
            nn.LeakyReLU(0.1),
            nn.Linear(dim_in // 2, dim_in_final),
            nn.LeakyReLU(0.1))

to

self.prd_cls_feats = nn.Sequential(
            nn.Linear(dim_in, dim_in_final),
            nn.LeakyReLU(0.1))
jz462 commented 5 years ago

Hi @juliuswang0728,

Good catch! Thank you for bringing this up, and yes you are right that there is a mismatch between the model structure and the provided weights. Since you have already spotted this bug, I suppose you have already trained a model with 1 fc layer, and I'm sure it won't be a huge difference. I'll definitely update the weight file once I have time and resources. Thanks again for mentioning this issue!

Ji

juliuswang0728 commented 5 years ago

Hi @jz462,

Thanks for the reply!

Just for sure, so the provided weights (trained_models/vg_VGG16/model_step62722.pth) were trained with 1FC model? I haven't had trained that myself, but will do :)

jz462 commented 5 years ago

@juliuswang0728 Well actually no matter what the provided weights say, you should always stick to the model provided in the code, since it is consistent with the model used on other datasets (VRD and OpenImages). So my suggestion is, do not change the code, re-train the model on VG if you have resources. If you have difficulty on training, I'll try my best to update the weights as soon as possible.

Ji

juliuswang0728 commented 5 years ago

Alright! Thanks for the clarification! I'll close the issue.