search

yuleiniu / cfvqa

[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias
Apache License 2.0
116 stars 14 forks source link

How to get the accuracy of the overall test set and the accuray in Y/N, other and number #10

Closed zpltys closed 2 years ago

zpltys commented 2 years ago

logs.txt I try to run the baseline as "python -m bootstrap.run -o cfvqa/options/vqacp2/smrl_baseline.yaml", and got the logs.txt. However, I cannot get the overall accuracy as similar in your paper (about 38.46), and how to get the accuracy of different question types (Y/N, other and number) from this log?

Should I run more epoch or change some super parameter?

yuleiniu commented 2 years ago

According to the description of result files, you can check the json file logs_val_oe.json to get the results of baseline. For example, for Y/N questions, the key is eval_epoch.perAnswerType.yes/no.

zpltys commented 2 years ago

Thanks for your reply!