ValueError('need at least one array to stack')

[makaveli10]

Hi, I am trying to train on a custom dataset my labels are in camera coordinate frame and looks like this

Car 0 0 -10 633 262 695 304 1.49 2.16 4.79 -1.17 0.99 35.98 3.11

However, after running python tools/create_data.py kitti --root-path ./data/kitti --out-dir ./data/kitti --extra-tag kitti, I see that both training/velodyne_reduced/*.bin and testing/velodyne_reduced/*.bin are empty. Also, the files in kitti_gt_database/*.bin are empty. I have very little clue about whats going wrong here because there was no error or info while creating the data from what I had.

Anyway, this is the what I get after running:

python tools/train.py configs/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py

File "/opt/conda/lib/python3.7/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.7/site-packages/mmdet/datasets/dataset_wrappers.py", line 178, in __getitem__
    return self.dataset[idx % self._ori_len]
  File "/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 433, in __getitem__
    data = self.prepare_train_data(idx)
  File "/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 227, in prepare_train_data
    example = self.pipeline(input_dict)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/compose.py", line 48, in __call__
    data = t(data)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/transforms_3d.py", line 332, in __call__
    ground_plane=ground_plane)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/dbsampler.py", line 243, in sample_all
    avoid_coll_boxes)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/dbsampler.py", line 323, in sample_class_v2
    sp_boxes = np.stack([i['box3d_lidar'] for i in sampled], axis=0)
  File "<__array_function__ internals>", line 6, in stack
  File "/opt/conda/lib/python3.7/site-packages/numpy/core/shape_base.py", line 422, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

Any help or pointers are most welcome.

[Resham-Sundar]

Hi, I am trying to train on a custom dataset my labels are in camera coordinate frame and looks like this

Car 0 0 -10 633 262 695 304 1.49 2.16 4.79 -1.17 0.99 35.98 3.11

However, after running python tools/create_data.py kitti --root-path ./data/kitti --out-dir ./data/kitti --extra-tag kitti, I see that both training/velodyne_reduced/*.bin and testing/velodyne_reduced/*.bin are empty. Also, the files in kitti_gt_database/*.bin are empty. I have very little clue about whats going wrong here because there was no error or info while creating the data from what I had.

Anyway, this is the what I get after running:

python tools/train.py configs/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py

File "/opt/conda/lib/python3.7/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.7/site-packages/mmdet/datasets/dataset_wrappers.py", line 178, in __getitem__
    return self.dataset[idx % self._ori_len]
  File "/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 433, in __getitem__
    data = self.prepare_train_data(idx)
  File "/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 227, in prepare_train_data
    example = self.pipeline(input_dict)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/compose.py", line 48, in __call__
    data = t(data)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/transforms_3d.py", line 332, in __call__
    ground_plane=ground_plane)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/dbsampler.py", line 243, in sample_all
    avoid_coll_boxes)
  File "/mmdetection3d/mmdet3d/datasets/pipelines/dbsampler.py", line 323, in sample_class_v2
    sp_boxes = np.stack([i['box3d_lidar'] for i in sampled], axis=0)
  File "<__array_function__ internals>", line 6, in stack
  File "/opt/conda/lib/python3.7/site-packages/numpy/core/shape_base.py", line 422, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

Any help or pointers are most welcome.

I am facing the same issue.Were you able to solve it?

[makaveli10]

@Resham-Sundar no, what dataset are you using?

[Resham-Sundar]

I am also using a custom dataset. And I have converted it into KITTI format.

[Tai-Wang]

Please make sure the camera calibration information is correct. KITTI pre-processing will use that to filter out point clouds outside of the front-view and produce the velodyne_reduced files.

[makaveli10]

@Tai-Wang I am trying to train on custom dataset(simulated):

1. Lidar points are saved in lidar coordinates (kitti format)
2. object location saved in camera coordinates (kitti format)
3. I have lidar & camera placed exactly on the same location and with same transforms. So, my Trv2c comes down to this as it only need to switch the corrdinate frame from lidar to camera.

   
   TR_velodyne = np.array([
                           [0, -1, 0],
                           [0, 0, -1],
                           [1, 0, 0]
   ])

Add translation vector from velo to camera. This is 0

because the position of camera and lidar is equal in our configuration.

TR_velodyne = np.column_stack((TR_velodyne, np.array([0, 0, 0])))

4. R0_rect is identity for me as I only have 1 camera. And, I have double checked the intrinsics of the camera which seems correct. But, ```tools/create_data.py``` discards all the lidar points for most of the samples.

[makaveli10]

@Tai-Wang You were right, there was an issue with my transformations particularly Tr_velodyne, which I seem to have fixed, can confirm with the below images:

but I still see the same issue, and this time when I run tools/create_data.py I have the velodyne_reduced/ files with lidar data but the files in kitti_gt_database are empty for some reason.

[Tai-Wang]

It's really strange because it seems that the annotation is correct as visualized in the mayavi. Have you ever checked the process of generating kitti_gt_database and is there anything abnormal?

[makaveli10]

@Tai-Wang Got it solved. Some of the point cloud samples had no points in the camera fov which caused the issue. Thanks.

[calmelo]

@Tai-Wang I am trying to train on custom dataset(simulated):

1. Lidar points are saved in lidar coordinates (kitti format)
2. object location saved in camera coordinates (kitti format)
3. I have lidar & camera placed exactly on the same location and with same transforms. So, my Trv2c comes down to this as it only need to switch the corrdinate frame from lidar to camera.

TR_velodyne = np.array([
                            [0, -1, 0],
                            [0, 0, -1],
                            [1, 0, 0]
])

# Add translation vector from velo to camera. This is 0 
# because the position of camera and lidar is equal in our configuration.
TR_velodyne = np.column_stack((TR_velodyne, np.array([0, 0, 0])))

4. R0_rect is identity for me as I only have 1 camera. And, I have double checked the intrinsics of the camera which seems correct. But, tools/create_data.py discards all the lidar points for most of the samples.

I only have 1 camera and 1 lidar too, so how do you fill your calib files? Could you share your calib files? If the transform matrix is not complete, there will be errors. Also, the files in kitti_gt_database/*.bin are empty, how do you resolve the problem? I will be very appreciated if you can help me. Thanks!

[calmelo]

@Tai-Wang Got it solved. Some of the point cloud samples had no points in the camera fov which caused the issue. Thanks.

How did you resolve the problem by change the camera 'fov'?