search

fanyix / STMN

[ECCV 2018] Video Object Detection with an Aligned Spatial-Temporal Memory
http://fanyix.cs.ucdavis.edu/project/stmn/project.html
Other
117 stars 16 forks source link

The results are not the same with the paper #2

Closed wangpichao closed 5 years ago

wangpichao commented 5 years ago

I used the provided stmn model to test on VID val dataset, and got the following results:Category AP airplane 0.814 antelope 0.788 bear 0.900 bicycle 0.772 bird 0.734 bus 0.838 car 0.591 cattle 0.861 dog 0.802 domestic_cat 0.894 elephant 0.796 fox 0.970 giant_panda 0.847 hamster 0.993 horse 0.832 lion 0.714 lizard 0.852 monkey 0.539 motorcycle 0.706 rabbit 0.811 red_panda 0.865 sheep 0.752 snake 0.584 squirrel 0.619 tiger 0.922 train 0.839 turtle 0.722 watercraft 0.656 whale 0.709 zebra 0.937

Mean AP: 0.789 Median AP: 0.807

Results:
   81.3629
   78.8245
   90.0161
   77.1704
   73.3689
   83.7963
   59.1274
   86.0662
   80.2445
   89.4197
   79.5920
   97.0293
   84.6849
   99.3268
   83.1807
   71.4291
   85.1871
   53.9491
   70.6138
   81.1190
   86.5491
   75.1594
   58.4391
   61.9035
   92.2042
   83.9041
   72.1838
   65.5909
   70.8652
   93.6773

   78.8662

From the results, we could see that the mAP is 78.9, which is 1.6 lower than the paper reported, but the Median ap is around the report. What are the possible reasons?

fanyix commented 5 years ago

Are you combining the detections of the STMN model and the RFCN model? Or the STMN model alone?

wangpichao commented 5 years ago

Are you combining the detections of the STMN model and the RFCN model? Or the STMN model alone?

I just use the default configurations and run run_dp.m. How could I combine them together using available codes? Thanks.

fanyix commented 5 years ago

Before you run run_dp.m, please make sure you have run both "CUDA_VISIBLE_DEVICES=0 th eval_detect_full.lua -model rfcn -model_path ../../dataset/ImageNetVID/models/rfcn.t7 -ckpt rfcn_eval" and "CUDA_VISIBLE_DEVICES=0 th eval_detect_full.lua -model stmn -model_path ../../dataset/ImageNetVID/models/stmn.t7 -ckpt stmn_eval"

Enlistedman commented 4 years ago

@wangpichao Hi!How to visualize and save the resulting images?Any help will be appreciated.