search

tusen-ai / SST

Code for a series of work in LiDAR perception, including SST (CVPR 22), FSD (NeurIPS 22), FSD++ (TPAMI 23), FSDv2, and CTRL (ICCV 23, oral).
Apache License 2.0
801 stars 102 forks source link

performance of centerpoint and PC-RCNN in your paper #56

Closed seonhoon1002 closed 2 years ago

seonhoon1002 commented 2 years ago

Hello, I have question about performance in FSD paper in table 1

Bellow is your table which guessed tested on Waymo V1.2.0 validation image

And this is the table in Centerpoint image

And this is two version of PV-RCNN paper image

I think CenterPoint was measured from Waymo V1.0

Could I ask how did you know what version of Waymo for applying the Centerpoint

Abyssaledge commented 2 years ago

CenterPoint uses WOD >= 1.2. Note that Level 1 and Level 2 are different. Performance of Level 1 only takes GT containing more than 5 points into account.

seonhoon1002 commented 2 years ago

So if you're right, did centerpoint enter the wrong performacne about "PV-RCNN". Because PVRCNN performances in LV2 in centerpoint are "vehicle: 65.4AP/64.8APH, pedestrian: 53.9AP/46.7APH" but performance in LV2 in original paper are vehicle: 68.98AP/68.41APH, pedestrian: 66.04AP/57.61APH

Abyssaledge commented 2 years ago

Sorry for late reply. Both of them are right. The reported performance of PVRCNN in many papers are different because of different re-implementation and different version of dataset. I recommend to refer to the updated performance in PVRCNN paper.

seonhoon1002 commented 2 years ago

OK I see, Thank you