Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
Apache License 2.0
2.39k
stars
248
forks
source link
low accuracy on VQAv2 test-std when reproducing prompt tuning experiments #418
Hi OFA team, lots of appreciation for the great work!
Recently i've been trying to reproduce the experiments in the paper OFA-prompt, and i got a base-size model with accuracy of 73.08 on test-dev split.
However the test accuracy dropped significantly on test-std set, in which i only got around 22.38(11189 out of 50000).
below is my training script:
during evaluating, i changed batch-size to 80 and ran single chunked file in #68 . I wonder whether batch-size and size of test-size affects. Thanks for your attention @JustinLin610
Hi OFA team, lots of appreciation for the great work! Recently i've been trying to reproduce the experiments in the paper OFA-prompt, and i got a base-size model with accuracy of 73.08 on test-dev split. However the test accuracy dropped significantly on test-std set, in which i only got around 22.38(11189 out of 50000). below is my training script:
during evaluating, i changed batch-size to 80 and ran single chunked file in #68 . I wonder whether batch-size and size of test-size affects. Thanks for your attention @JustinLin610