Cyanogenoid / pytorch-vqa

Strong baseline for visual question answering
238 stars 97 forks source link

Fix a bug in accuracy computation and make apply_attention nicer. #18

Closed guoyang9 closed 5 years ago

guoyang9 commented 5 years ago

The accuracy from the computation of the acc_tracker is slightly better than that of the official evaluation method. One reason for this is the overall accuracy is not constant with the mean of all the accuracy from each sample when the number of all samples cannot be divided by batch_size.

For example, Let's suppose there are 7 samples and the batch_size is 3. The corresponding accuracy is [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.3]. Following the accuracy computation in this code, the overall accuracy is (0.1+0.1+0.3)/3=5/30=1/6. However, the correct overall accuracy should be (0.1*6+0.3)/7=9/70. This could result the former being larger than the latter.

guoyang9 commented 5 years ago

I changed the apply_attention() as shown in this branch. I found this is easier to understand and implement.