Closed bhaskar-anand-iith closed 2 years ago
@bhaskar-anand-iith Someone seems reported a similiar issue before, it's a bug about COCO eval API, so you may comment these code, as they are not useful nowadays.
Even while testing I am getting the same error. Please let me know which part is to be commented, since eval.py has fully coco eval API only.
(ssdenv) bhaskar@bhaskar:/media/bhaskar/Data_Disk/object_detection/SE-SSD/tools$ python test.py True Namespace(checkpoint='epoch_58.pth', config='../examples/second/configs/config.py', eval=None, eval_id=None, json_out='json_out.json', launcher='none', local_rank=0, out='out.pkl', show=False, tmpdir=None, txt_result=True, vis_id=None) [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ] 3765/3768, 52.6 task/s, elapsed: 72s, ETA: 0s/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (45) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (54) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (60) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (40) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (50) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (48) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (6) < 2 * SM count (164) will likely result in GPU under utilization due to low occupancy. warn(NumbaPerformanceWarning(msg)) /media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/core/typed_passes.py:330: NumbaPerformanceWarning: The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.
To find out why, try turning on parallel diagnostics, see https://numba.readthedocs.io/en/stable/user/parallel.html#diagnostics for help.
File "../det3d/datasets/utils/eval.py", line 324: @numba.jit(nopython=True, parallel=True) def box3d_overlap_kernel(boxes, qboxes, rinc, criterion=-1, z_axis=1, z_center=1.0): ^
state.func_ir.loc))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (54) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (60) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (45) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (40) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (50) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (48) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numba/cuda/compiler.py:726: NumbaPerformanceWarning: Grid size (6) < 2 SM count (164) will likely result in GPU under utilization due to low occupancy.
warn(NumbaPerformanceWarning(msg))
Traceback (most recent call last):
File "test.py", line 269, in
Solved it..
Follow: https://github.com/Vegeta2020/SE-SSD/issues/81#issuecomment-1102136064
I am getting following error after completing the 60 epochs,
File "train.py", line 118, in
main()
File "train.py", line 115, in main
train_detector(model, datasets, cfg, distributed=distributed, validate=args.validate, logger=logger,)
File "/media/bhaskar/Data_Disk/object_detection/SE-SSD/det3d/torchie/apis/train_sessd.py", line 323, in train_detector
trainer.run(data_loaders, cfg.workflow, cfg.total_epochs, local_rank=cfg.local_rank)
File "/media/bhaskar/Data_Disk/object_detection/SE-SSD/det3d/torchie/trainer/trainer_sessd.py", line 469, in run
epoch_runner(data_loaders[2], *kwargs)
File "/media/bhaskar/Data_Disk/object_detection/SE-SSD/det3d/torchie/trainer/trainer_sessd.py", line 407, in val
resultdict, = self.data_loader.dataset.evaluation(predictions, output_dir=self.work_dir)
File "/media/bhaskar/Data_Disk/object_detection/SE-SSD/det3d/datasets/kitti/kitti.py", line 157, in evaluation
result_coco_dict = get_coco_eval_result(gt_annos, dt_annos, self._class_names, z_axis=z_axis, z_center=z_center)
File "/media/bhaskar/Data_Disk/object_detection/SE-SSD/det3d/datasets/kitti/eval.py", line 739, in get_coco_eval_result
z_center=z_center,
File "/media/bhaskar/Data_Disk/object_detection/SE-SSD/det3d/datasets/kitti/eval.py", line 437, in do_coco_style_eval
min_overlaps[:, i, j] = np.linspace(overlap_ranges[:, i, j])
File "<__array_function__ internals>", line 6, in linspace
File "/media/bhaskar/Data_Disk/object_detection/ssdenv/lib/python3.7/site-packages/numpy/core/function_base.py", line 120, in linspace
num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer
Please help if you faced such issue..