Problem of evaluation on temporal settings.

[weimengchuan]

Thanks for yout great work!

I am doing evaluation on the temporal setttings, but obtain lower performance than that you have reported in github.

My command:

bash tools/dist_test_map.sh projects/configs/maptr/maptr_tiny_r50_24e_t4.py ckpts/epoch_24.pth 1
bash tools/dist_test_map.sh projects/configs/maptr/maptr_tiny_r50_24e_bevformer_t4.py ckpts/maptr_tiny_r50_24e_t4.pth 1

My results:

• maptr_tiny_r50_24e_t4.py My result is 39.2 mAP, which is lower than 51.3.

• maptr_tiny_r50_24e_bevformer_t4.py My result is 39.7 mAP, which is lower than 53.3.

Another problem: When I download the pth files, I find that the configuration file name does not correspond to the weight file name (as is shown in My command). I try to exchange the order of pth files, and run following commands, but mAPs are almost zero.

Your help is greatly appreciated.

[iuhiyuh]

set "video_test_mode=True"

[huiyiygy]

Hi, I encountered the same issue. Follow @iuhiyuh method to modify the configuration model file: maptr_tiny_r50_24e_t4.py maptr_tiny_r50_24e_bevformer_t4.py set "video_test_mode=True" I could achieve the same mAP as in the README.md