search

MILVLG / openvqa

A lightweight, scalable, and general framework for visual question answering research
Apache License 2.0
318 stars 64 forks source link

why the result is 0.01 when I validate #43

Closed heyumeng123 closed 4 years ago

heyumeng123 commented 4 years ago

I run your model about mfb.When I train,every step is done according to your requirements.The input is"python run.py --RUN='train' --MODEL='mfb' --DATASET='vqa' " And we can see the results.The accuracy is about 65%. when I validate ,The input is "python run.py --RUN='val' --MODEL='mfb' --DATASET='vqa ' --CKPT_V=7153401 --CKPT_E=13" 7153401 is the version about loaded model.But the results is 0.01. Why it is 0.01. I dont know what's wrong.

yuzcccc commented 4 years ago

The phenomenon is really odd, and we have rerun the command just as you provided (we only trained for 1 epoch) but the result obtained by manual validation is consistent with the validation during training. Do you use other augment when during training or testing, e.g., accu or multiple GPUs?

heyumeng123 commented 4 years ago

After the dataset is downloaded, I start training.The dataset is what you presented on the web. I dont have the ability to change the code, so I didn't use other augment during training or testing.Now I have trained it again.But the result is 0.01

MIL-VLG commented 4 years ago

This is caused by a bug indeed. We have fixed the problem in #44