Wrong 3d IOU calculation produced by OpenCV

[zye1996]

Hi I am experimenting the box merge and score method used in the paper and I am using the functions provided to calculated 3D IOU metric while it seems not being accurate for the function overlapped_boxes_3d.

Here is my code:

bbox1 = torch.tensor([0, 0, 0, 4, 4, 4, 0])
bbox2 = torch.tensor([0, 0, 0, 2, 2, 2, 0])
bbox1_corner = boxes_3d_to_corners([bbox1])
bbox2_corner = boxes_3d_to_corners([bbox2])
overlapped_boxes_3d(bbox1_corner[0], bbox2_corner)

It seems that OpenCV is not producing the correct result for cv2.fillPoly() as when I tested the following code:

buf = np.zeros((4,4))
cv2.fillPoly(buf, np.array([[[3, 3], [3, 1], [1, 1], [1, 3]]]), color=1)

the result is

array([[0., 0., 0., 0.],
       [0., 1., 1., 1.],
       [0., 1., 1., 1.],
       [0., 1., 1., 1.]])

The result should obviously be 0.125 while it yields 0.28125. The function overlapped_boxes_3d_fast_poly yields correct result as 0.125

[WeijingShi]

Hi @zye1996, yes, the OpenCV implementation is an approximation method, the overlapped_boxes_3d_fast_poly() use the shapely library to solve it accurately. If the OpenCV method is still desired, we can increase the approximation accuracy by multiplying an expanding factor (say 100) to the box corners.

[zye1996]

Hi @zye1996, yes, the OpenCV implementation is an approximation method, the overlapped_boxes_3d_fast_poly() use the shapely library to solve it accurately. If the OpenCV method is still desired, we can increase the approximation accuracy by multiplying an expanding factor (say 100) to the box corners.

Hi Weijing thank you for your quick response! And there is another question I have for the code, according to the paper box score with occlusion should be updated in NMS as

I found the relative code in bboxes_nms_uncertainty function to calculate the second multiplier which is the sum of 3D IOU score, and the occlusion score is then calculated in run.py for the term o+1. By my understanding, the code at line 161 should assign scores[i] = np.sum(scores[(i+1):][valid][remove_overlap]*boxes_mean_overlap) but it is scores[i] += np.sum(scores[(i+1):][valid][remove_overlap]*boxes_mean_overlap) in the file.

I am not sure if I missed anything so can you please let me know if I am wrong?

[WeijingShi]

Hi @zye1996, the difference is on how to handle the i-th box itself. i-th box is also a member of the overlapped boxes. We can ignore it by using:

scores[i] = np.sum(scores[(i+1):][valid][remove_overlap]*boxes_mean_overlap)

But in the case where there are no other overlapped boxes, we get score[i]=0.

Instead, we can include i-th box's score in the sum:

scores[i] = score[i] + np.sum(scores[(i+1):][valid][remove_overlap]*boxes_mean_overlap)

Technically, we also need to compute i-th box's IoU with the median_box as the weight factor, so that it follows the exact equation, though.

[zye1996]

Hi @zye1996, the difference is on how to handle the i-th box itself. i-th box is also a member of the overlapped boxes. We can ignore it by using:

scores[i] = np.sum(scores[(i+1):][valid][remove_overlap]*boxes_mean_overlap)

But in the case where there are no other overlapped boxes, we get score[i]=0.

Instead, we can include i-th box's score in the sum:

scores[i] = score[i] + np.sum(scores[(i+1):][valid][remove_overlap]*boxes_mean_overlap)

Technically, we also need to compute i-th box's IoU with the median_box as the weight factor, so that it follows the exact equation, though.

Thank you so much, your answer solves my question!