search

MILVLG / mcan-vqa

Deep Modular Co-Attention Networks for Visual Question Answering
Apache License 2.0
442 stars 88 forks source link

question regarding test split #24

Closed fawazsammani closed 4 years ago

fawazsammani commented 4 years ago

Hello! Thanks for sharing your code and brilliant work. I'd like to ask about the evaluation on test-std and test-dev. Is there any way to know the number of epochs needed for training, since evaluation is not available. I've seen that in your case, you used the same number of training epochs (13). But I assume that since the training data is largely increased (since for evaluation on test set requires training on 'train+val+vg' sets), the number of epochs required for convergence will also increase. Or do you evaluate a number of epochs on the online server to see which performs better?

Thanks for sharing your suggestion.

Regards

MIL-VLG commented 4 years ago

We have evaluated the test-dev results near the 13th epoch (e.g., 12th, 14th, 15th). The 13th is overall the best from our observations

fawazsammani commented 4 years ago

Thanks a lot.