Training and Evaluation setting

[drigoni]

Dear authors,

from the config files I see that for the Detectron2 version the splits used are the following: DATASETS: TRAIN: ("visual_genome_train", "visual_genome_val") TEST: ("visual_genome_test",) while for Caffe: DATASETS: TRAIN: ("visual_genome_train",) TEST: ("visual_genome_val",) So, I would like to ask if:

1. do the model results shown in the table for the Caffe version actually refer to the validation split while those of Detectron2 refer to the test split? Or is there an error in the config files?
2. does the pre-trained model you load based on the Detectron2 version get trained in both training and evaluation data?

Best regards, Davide

[1219521375]

Yes. The results about caffe version refer to the validation split, and the detectron2 version refer to the test split. For caffe version, we follow the training settings in original caffe implements, which is trained in the train split and tested in the val split. For detectron2 version, we follow the training settings in grid-feats-vqa, which is trained in both train split and val split, then tested in test split.