ZeroDivisionError

[LeonRuddat]

Hi everyone,

First of all, I want to thank you for publishing this wonderful dataset. I am currently working on reproducing the radar results (5 scans) from the paper. When I use the evaluation tool from your repo, I sometimes (not often though) get a ZeroDivisionError:

Traceback (most recent call last):
  File "/home/leon/Projects/OpenPCDet/tools/eval_delft.py", line 21, in <module>
    results = evaluation.evaluate(
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/evaluate.py", line 44, in evaluate
    evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class))
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 741, in get_official_eval_result
    mAPbbox, mAPbev, mAP3d, mAPaos, mAPbbox_R40, mAPbev_R40, mAP3d_R40, mAPaos_R40 = do_eval(
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 665, in do_eval
    ret = eval_class(gt_annotations, dt_annotations, current_classes, difficulties, 1,
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 533, in eval_class
    rets = calculate_iou_partly(dt_annotations, gt_annotations, metric, num_parts)
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 432, in calculate_iou_partly
    overlap_part = bev_box_overlap(gt_boxes, dt_boxes).astype(
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 152, in bev_box_overlap
    r_iou = rotate_iou_eval(boxes, q_boxes, criterion)
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/rotate_iou_cpu.py", line 269, in rotate_iou_eval
    iou[n, k] = instance_iou_eval(query_boxes[k], boxes[n], criterion)
ZeroDivisionError: division by zero

I found out that the error occurs in the _sort_vertex_in_convexpolygon function in the _rotate_ioucpu.py. Somehow d = math.sqrt(v[0] * v[0] + v[1] * v[1]) becomes zero. Here is the skript that i wrote using your example:

from vod.evaluation import Evaluation
import os
import wandb
from tqdm import tqdm
#wandb.init("Eval Delft", name="EvalDelft_2 ", reinit=True)
# When the instance is created, the label locations are required.
evaluation = Evaluation(test_annotation_file="/home/leon/Projects/OpenPCDet/data/kitti/training/label_2")

# Using the evaluate method, the model can be evaluated on the detection labels.
EPOCHS = []
default_dir = "/home/leon/Projects/OpenPCDet/output/kitti_models/pointpillardelftradar/default/eval/eval_with_train"
for _, dirs, _ in os.walk(default_dir):
      for dir in dirs:
            if dir.startswith("epoch"):
                  EPOCHS.append(dir)

EPOCHS.sort()
for epoch in tqdm(EPOCHS):
      results = evaluation.evaluate(
          result_path=os.path.join(default_dir, epoch, "val/final_result/data"),
          current_class=[0, 1, 2])

      print("Results: \n"
            f"Entire annotated area: \n"
            f"Car: {results['entire_area']['Car_3d_all']} \n"
            f"Pedestrian: {results['entire_area']['Pedestrian_3d_all']} \n"
            f"Cyclist: {results['entire_area']['Cyclist_3d_all']} \n"
            f"mAP: {(results['entire_area']['Car_3d_all'] + results['entire_area']['Pedestrian_3d_all'] + results['entire_area']['Cyclist_3d_all']) / 3} \n"
            f"Driving corridor area: \n"
            f"Car: {results['roi']['Car_3d_all']} \n"
            f"Pedestrian: {results['roi']['Pedestrian_3d_all']} \n"
            f"Cyclist: {results['roi']['Cyclist_3d_all']} \n"
            f"mAP: {(results['roi']['Car_3d_all'] + results['roi']['Pedestrian_3d_all'] + results['roi']['Cyclist_3d_all']) / 3} \n"
            )

Except for the accidentally swapped input for the Eval function (has been merged into the master in the meantime) I have not changed anything in the code. The training process itself runs smoothly. (I followed your guidelines → not quite reached the benchmark though). Not sure if I'm doing something wrong.... Have any of you ever had such a problem, or any idea what the problem could be ? Thanks in advance! Best Regards Leon

[BalazsSzekeres]

Hi @LeonRuddat

Could you share the files for which the zero-division occurred?

We have a fix in the pipeline and want to make sure it fixes the problem in your case as well.

[LeonRuddat]

Hi @BalazsSzekeres

of course, here is a drive link to the epoch where the error occurred. In total, the error has occurred 4 times so far. Unfortunately, I did not save the data of the other three times. But the error should be of the same origin, because it always appeared in the same place.

It would be nice if you could let me know what exactly triggered the error. Just out of interest :)

https://drive.google.com/drive/folders/1v6LBOxWSKki8mk8WeySdlXrYnmyFqUnf?usp=sharing

Leon

[BalazsSzekeres]

Hi Leon,

Can confirm the issue is now fixed.

Debugging was a bit more annoying as we compile the code for better performance using the Numba package. Within the rotation calculation file, there is a method that handles polygons. In case the polygon is of poor quality, a distance function returns zero, which is then used for division, thus causing the exception.

Exception handling has been now put in place.

The updated pip package you can find here with version number 1.0.3: https://pypi.org/project/vod-tudelft/

[LeonRuddat]

Allright, thanks for your help and the detailed explanation. It works for me now too.