search

open-mmlab / mmdetection3d

OpenMMLab's next-generation platform for general 3D object detection.
https://mmdetection3d.readthedocs.io/en/latest/
Apache License 2.0
5.29k stars 1.54k forks source link

model training with visual-SLAM #45

Closed niezhongliang closed 4 years ago

niezhongliang commented 4 years ago

hi, thanks for the work. I am trying to use the mmdet3d to detect the point cloud from the vSLAM without camera image. How could I train the model? Are there any suggestions? Because I find the tools/create_data.py has many funcs relied on the camera 2d boxes.

Thanks

Tai-Wang commented 4 years ago

Actually you can first organize your point cloud data like our supported datasets (like training/velodyne and testing/velodyne in KITTI), and rewrite your own dataset converter (like kitti_converter.py/nuscenes_converter.py) and APIs (like kitti_dataset.py/nuscenes_dataset.py). Image related steps in the create_data.py like export_2d_annotation can be omitted.

For models, you can borrow our provided models/configs that use only point cloud data (like pointpillars and second) and just adjust some details to fit your case.

niezhongliang commented 4 years ago

Thanks! I wil try it.

niezhongliang commented 4 years ago

Actually you can first organize your point cloud data like our supported datasets (like training/velodyne and testing/velodyne in KITTI), and rewrite your own dataset converter (like kitti_converter.py/nuscenes_converter.py) and APIs (like kitti_dataset.py/nuscenes_dataset.py). Image related steps in the create_data.py like export_2d_annotation can be omitted.

For models, you can borrow our provided models/configs that use only point cloud data (like pointpillars and second) and just adjust some details to fit your case.

1596100825(1)

Thanks for your help at first. Could you tell me why I need to do this conversion here? originally, bbox_3d is got by annos, and why I need to convert.

In my situation, I don't have calibration information, thus when I do this convert, the bbox_3d results will be changed later, and seems will be not compatible with the gts.

Tai-Wang commented 4 years ago

Actually you can first organize your point cloud data like our supported datasets (like training/velodyne and testing/velodyne in KITTI), and rewrite your own dataset converter (like kitti_converter.py/nuscenes_converter.py) and APIs (like kitti_dataset.py/nuscenes_dataset.py). Image related steps in the create_data.py like export_2d_annotation can be omitted. For models, you can borrow our provided models/configs that use only point cloud data (like pointpillars and second) and just adjust some details to fit your case.

1596100825(1)

Thanks for your help at first. Could you tell me why I need to do this conversion here? originally, bbox_3d is got by annos, and why I need to convert. In my situation, I don't have calibration information, thus when I do this convert, the bbox_3d results will be changed later, and seems will be not compatible with the gts.

It is because the annotations given by KITTI are in camera coordinates. You can refer to nuScenes and Lyft, where this problem does not exist. Basically simliar conversions are data-specific, and please feel free to adjust them to fit your case.