search

Scalsol / mega.pytorch

Memory Enhanced Global-Local Aggregation for Video Object Detection, CVPR2020
Other
565 stars 115 forks source link

Testing on 'test' dataset instead of 'validation' dataset? #24

Closed Hastyrush closed 4 years ago

Hastyrush commented 4 years ago

Hello! Thank you for your generous release of the source code on MEGA.

I was just wondering, the config files show that you tested the model with only the validation dataset, is there any reason why you did not test with the 'test' dataset of both VID and DET dataset? If I wanted to do that, will a simple change in the config file be enough? Or do I have to alter the 'ImageSets' folder to provide a list of files that I want to test with? Thanks a lot!

Scalsol commented 4 years ago

Hi, thank you for your interest! Testing on the validation dataset is adopted in all previous method so we just follow this. But to be honest, testing on the test dataset is impossible because no public annotation is provided, and the test server is closed. If you just want to see the result for visualization on test dataset, you could generate a new txt file follow VID_val_videos.txt. The meaning of the 4 strings are video folder, no meaning(but I still keep it), frame number, video length Hope this helps.

Hastyrush commented 4 years ago

Hi, thanks for the clarification!

When you mention see the result for visualization, do you mean the raw prediction of the test dataset without any comparison to ground truth? (I.e. bbox of each object, class in the form of a pickle file). Thanks in advance!

Scalsol commented 4 years ago

Yes, actuallty I would add demo for visualization these days :)

Hastyrush commented 4 years ago

That would be great! Thanks, will be closing this issue