Question: How to perform inference from sensor_msgs/msg/PointCloud2 messages

[tleyden]

I'm trying to perform inference with sensor_msgs/msg/PointCloud2 messages with the following structure:

---
header:
  stamp:
    sec: 1591579194
    nanosec: 532001792
  frame_id: lidar_front
height: 1
width: 4314
fields:
- name: x
  offset: 0
  datatype: 7
  count: 1
- name: y
  offset: 4
  datatype: 7
  count: 1
- name: z
  offset: 8
  datatype: 7
  count: 1
- name: intensity
  offset: 16
  datatype: 2
  count: 1
- name: timestamp
  offset: 24
  datatype: 8
  count: 1
is_bigendian: false
point_step: 32
row_step: 138048
data: [41, 80, 35, 65, 251, 212, 240, 183, 63, 95, 230, 191, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 148, 252, 34, 65, 54, 27, 54, 190, 62, 242, 229, 191, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 127, 168, 34, 65, 194, 198, 181, 190, 142, 150, 229, 191, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 226, 89, 34, 65, 144, 36, 8, 191, 80, 84, 229, 191, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '...']
is_dense: true

If I converted this into the same point cloud format that pcdet expects, would this be enough to perform inference and generate 3D bounding boxes? Any examples or pointers?

[AndyYuan96]

not enough. As kitti dataset is a little different from other dataset like nuscense, waymo, kitti's 3d label is in camera coordinate, and the anchor is configured according to kitti dataset. so you need to pay attention to following things:

1. let's kitti dataloader to load your data(easy)
2. when using output bounding boxs, you should plus height/2 in box's center in height dim(related to kitti dataset).

[TillBeemelmanns]

Your PointCloud2 message format is an ordered point cloud. You can use pcl::fromROSMsg and pcl_conversions to convert the point cloud into a list of points in format XYZIR like the KITTI dataset.

[undefinedzero]

You can use ros_numpy to transform the PointCloud2 ROS message into a NumPy array, then convert the array into the format that PCDet expects (according to __getitem__ function in pcdet/datasets/kitti/kitti_dataset.py). If you want boxes in LiDAR coordinate for evaluation, you can modify the generate_prediction_dict function which will be called after the model inference.

[muzi2045]

refer this PR:

https://github.com/open-mmlab/OpenPCDet/pull/193

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.