search

gdlg / panoramic-object-detection

Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360° Panoramic Imagery
https://gdlg.github.io/panoramic
Other
71 stars 18 forks source link

Benchmark Leaderboard #6

Open plaut opened 4 years ago

plaut commented 4 years ago

Do you maintain a leaderboard for 3D object detection on your benchmark synthetic dataset?

Where can I find instructions for submitting my results?

gdlg commented 4 years ago

Hello, I haven’t had any interest yet in a leaderboard but I am happy to make one if that’s of interest to you.

Please email your results on the test dataset (*.txt files, same format as the KITTI benchmark) to me and I will run the evaluation against the groundtruth and email back the results to you.

plaut commented 4 years ago

Thank you. I have a few more questions about the dataset:

  1. In the validation set there are no DontCare areas, yet there are clearly visible cars that are not labeled at all (and they are larger than 25 pixels). Is this also true for the test set?

  2. It appears that the horizontal center line in the panorama images is not the spherical projection's horizon, it is shifted by 10 degrees. Can you confirm?

gdlg commented 4 years ago

  1. This is a problem with the dataset. The static vehicles in the simulation were not captured in the list of vehicles (something that I thought that I had fixed). Thanks for spotting that. I can add DontCare areas but alas it is going to take some time to do so.

  2. Yes, that’s correct. The horizon is 88 pixels from the top. Numerically, I get 10.9°:

    shift_pix = 300/2 - 88
angular_precision = 1024/pi
shift_rad = shift_pix / angular_precision
shift_deg = shift_rad * 180/pi
plaut commented 4 years ago

If you can upload an updated dataset enough time before March 5, 2020 I would be happy to test on it and compare my results to your method.