search

HKUST-Aerial-Robotics / Stereo-RCNN

Code for 'Stereo R-CNN based 3D Object Detection for Autonomous Driving' (CVPR 2019)
MIT License
690 stars 177 forks source link

About the performance for Kitti testset. #57

Closed tjulyz closed 4 years ago

tjulyz commented 4 years ago

Hi, I have carefully read the paper and the performance in your paper. From the leaderboard in Kitti, the performance is about 23% for hard subset. image But in your paper, the performance is about 28% for hard testset. So could you please tell how to improve the test performance from 23% to 28%? Have you used all the training data (i.e., training subset and validation subset) for training the final network when reporting the test performance? Thanks a lot!

PeiliangLi commented 4 years ago

Hi, seems like the KITTI has changed the benchmark recently. You can check the details in the kitti official website, and previous results are in:http://www.cvlibs.net/datasets/kitti/backups/2019_10_08_10_46_41_object_3d.html

tjulyz commented 4 years ago

Thanks a lot! So have you used all the training data (7000+) to get the final model for testset or just used 3000+ training data?

PeiliangLi commented 4 years ago

Hi, @tjulyz sorry for the late reply. I use all the training data to get the kitti test results and the epoch should also be reduced (might be around 8 epochs). A simple Gaussian noise was also added to the input image. But I am not sure if this helps because we didn't do an ablation study due to the submission time limits.

tjulyz commented 4 years ago

Thanks a lot! Could you please tell how to train this final model? Have you change the learning rate or using finetuning?

tjulyz commented 4 years ago

Hi, I have tried the strategies you mentioned above (8 epochs and gaussian noise), but I still cannot reproduce the performance on test set (I can only achieve 18+% for hard set). Could you please provide more details? Thanks a lot!

PeiliangLi commented 4 years ago

Hi @tjulyz, I trace back to my previous experiment code and upload it to the exp branch. It should be the version for test server submission. Just run ./train.sh and test.sh to try it out. I may not track the issue very frequently, feel free to drop me an email if you need immediate responses. Thank you.

tjulyz commented 4 years ago

I will try it later. Thank you very much!