search

open-mmlab / OpenPCDet

OpenPCDet Toolbox for LiDAR-based 3D Object Detection.
Apache License 2.0
4.57k stars 1.28k forks source link

How does the refinement network of PVRCNN++ work when there is no valid ROI? #764

Closed Kemo-Huang closed 2 years ago

Kemo-Huang commented 2 years ago

When there is few ROI from the RPN, "vector pool local interpolate" module will fail due to no support xyz or support feature.

However, it is very likely that there is no vehicle in the test data so that the RPN won't generate enough foreground ROI for refinement.

Specific error: File "/home/kemo/Github/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/pointnet2_modules.py", line 230, in forward near_known_xyz = support_xyz[idx.view(-1, 3).long()].view(-1, 3, 3) # ( (M1 + M2 ...)*num_total_grids, 3) IndexError: index is out of bounds for dimension with size 0

How to fix it?

sshaoshuai commented 2 years ago

Generally we will have several faked RoIs from proposal_layer, the RCNN heads will predict low confidence for these faked RoIs.

For your case, is it the problem of empty keypoints? I didn't meet this error before, maybe the SPC layer didn't find any points within the neighboring regions of RoIs. So you get empty support_xyz tensor.

You need to check whether the following warning is printed and maybe you need to fake some keypoints in this place to skip this error:

https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/models/backbones_3d/pfe/voxel_set_abstraction.py#L109-L119