search

WeijingShi / Point-GNN

Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud, CVPR 2020.
MIT License
523 stars 114 forks source link

a question about the precision of val and test #35

Open tyjiang1997 opened 3 years ago

tyjiang1997 commented 3 years ago

Hello, your research is very good. But I have a question about the accuracy of val and test. As you mentioned in your article, the accuracy of PointGNN for car on val-split is 78.40, but your accuracy on test is as high as 79.47. test: image val: image However, I made a comparison with other articles, and I found that the accuracy of most articles on val is positively related to the accuracy on test. If the accuracy is 79.5 on test, it is generally 79.8 on val. But recently my experiment is similar to yours. I got a lower AP, on val but a better result on test. Can you explain this reason? Thank you very much

WeijingShi commented 3 years ago

Hi @LittleYuanzi , Thank you for your interest. The ablate study on paper is all done using KITTI's original 11 recall positions. However, KITTI decides to change its test to 40 recall positions last year. So one possibility is that it impacts the comparison between the leaderboard and ablation study.