search

hustvl / AziNorm

AziNorm: Exploiting the Radial Symmetry of Point Cloud for Azimuth-Normalized 3D Perception, CVPR 2022.
MIT License
53 stars 2 forks source link

some question about result #2

Open AndyYuan96 opened 2 years ago

AndyYuan96 commented 2 years ago
  1. why your second baseline's all class result is lower compared to openpcdet's result, all class ap lower more than 2%, especially for ped( l1ap lower 7%)? small batch size will get better result, but don't have so much different, even I use 8 v100 32g to train, I don't have so much difference with openpcdet's result, did you use different config like bigger voxel size?
  2. why not compared with centerpoint, centerpoint just use centerhead as anchorhead, it don't have too much difference with Second, but it did give very much improvement for cyc and ped, does patch is still useful?
outsidercsy commented 2 years ago

Thanks for the questions.

  1. We start this work in January, 2021. Our code is based on a much older version (v0.4.0) of OpenPCDet (please refer to this git commit https://github.com/open-mmlab/OpenPCDet/tree/a7cf5368d9cbc3969b4613c9e61ba4dcaf217517). OpenPCDet is being continually updated and has come to v0.5.2 with improved implementation. The SECOND baseline of OpenPCDet v0.4.0 is lower than that of OpenPCDet v0.5.2. And it's worth noting that, even though AziNorm-based SECOND is based on a poorer version of OpenPCDet, it still significantly outperforms SECOND based on the up-to-date OpenPCDet. We will provide the code and models of both SECOND and AziNorm-based SECOND for comparison.

  2. For the same reason, at the time we conducted experiments, CenterPoint was not supported by OpenPCDet yet. (OpenPCDet v0.5.0 starts to support CenterPoint). We validated AziNorm based on available SECOND and PV-RCNN. The improvement of AziNorm comes from normalizing point clouds along the radial direction, instead of the design of patch (refer to Tab.6 of the paper). It’s intuitive that leveraging the radial symmetry is beneficial, no matter whether centerhead or anchorhead is adopted. We think AziNorm can also improve CenterPoint. If time permits, we will consider migrating the code into the newest version of OpenPCDet and including experiments about CenterPoint.

AndyYuan96 commented 2 years ago

1.openpcdet v0.5.2's improvement come from use gtaug from all data, rather than 10% data in v0.4.0, so v0.4.0 and v0.5.2 don't have too much different. 2.I know the AziNorm's improvement come from normalize along the radial direction, sorry for only saying patch。I think the implementation of operation that patch and then normalize is not hard, so why not use centerpoint's official repo to compare centerpoint and centerpoint with AziNorm, I just think compare with sota is more convincing.

outsidercsy commented 2 years ago
  1. Our experiments are based on the same augmentation of v0.4.0. Thus the baseline is lower.
  2. Thanks for the advice. It's OK to integrate AziNorm into centerpoint's official rep. Experiments of SECOND and PV-RCNN are both based on OpenPCDet. Working on a unified and well-developed codebase can reduce the workload.
jihanyang commented 2 years ago

Maybe I can help to answer this question. OpenPCDet changes to use two return data of Waymo in v0.5.0. The performance difference of 1 return and 2 return results can be found in the PVRCNN++ paper. @AndyYuan96

@outsidercsy Looking forward to see your code, and I think it would be much better to also provide results with OpenPCDet >= v0.5.0 since the 2return results of Waymo is more commonly used in recent works. I also worry about the performance mismatch before see this issue.