search

facebookresearch / VMZ

VMZ: Model Zoo for Video Modeling
Apache License 2.0
1.04k stars 156 forks source link

the test accuracy of the same test dataset is different when using the same model #53

Open leluoye opened 5 years ago

leluoye commented 5 years ago

Hi, @dutran , thank you for your great work. I finetuned the r2plus1d model on my own dataset using train_net.py, then, I got the best test accuracy 0.72 and the corresponding model r2plus1d_3.mdl. However, when I use the same test dataset and the r2plus1d_3.mdl to run test_net.py, the test accuracy is low. It is about 0.2. And I also tried to extract the features using extract_features.py and then got the test accuracy using dense_prediciton_aggregation.py. The test accuracy is low too, it is at most 0.12. It makes me feel confused. Why is the test accuracy so different? I know the value of the decode_type may influence the test accuracy, but I wonder if there are any other reasons that could affect the test accuracy? Could you give me some advice? Thank you.

dutran commented 5 years ago

It is hard to tell why it gives you different performance since you did not provide enough data/info here. But it's worth for you to check if you have the same hyper-parameter settings with train_net vs. test_net and featture_extraction?

leluoye commented 5 years ago

Thank you for your response. Follow the guideline, the .csv file of input data in train_net and test_net is "org_video, label", the .csv file of input data in feature_extraction is "org_video, label, start_frm, video_id". Then, I create lmdb file. The hyper-parameters setting with train_net, test_net and feature_extraction are shown in the picture. image

I was checked the hyper-parameter, but I could not find anything wrong. Thank you.

nicebaby commented 5 years ago

@leluoye Hi，does it work for you?