About the point cloud data input format of the function "information_detector()" during inference

[lwplw]

from mmdet3d. apis import init_ model, inference_ detector

result, _ = inference_ detector(model, point_cloud)

Can the data input of the function "information_detector()" be in other forms besides the point cloud file path? If I want to input the point cloud data directly, is there any other suitable way?

[ZCMax]

Of course,inferecne_detector supports input the point_cloud data directly (for example: numpy array format), more details can be seen here

[lwplw]

Of course, supports input the point_cloud data directly (for example: numpy array format), more details can be seen hereinferecne_detector

Thank you very much for your answer.

I read the point cloud data from rosbag and convert it to numpy format for topic, msg, t in bag_data: pointcloud_list = point_cloud2.read_points_list(msg) pointcloud_np = np.array(list(pointcloud_list)) ... modelresult, = inference_detector(model, pointcloud_np)

After being sent to the network, the following error occurred:

[lwplw]

I checked the 609 lines of the "transforms_3d. Py" file at the error location, and printed and viewed the data contents of "input_dict ['points']" and "rot_mat_t". The data transmitted through the file path and directly are as follows:

It seems that the data type of "input_dict ['points']" is just numpy array, and the way to read the file path is not just this. I'm not sure if the format of the data I passed in is correct.

[lwplw]

I went on to retrospectively enter the data format, and both looked the same as the definition of dict on lines 106 to 139 in the "inference.py" file:

[lwplw]

The "def inference_detector (model, pcd)" of the file "inference.py": the function can see that when the input data is not "str", the data is loaded using "LoadPointsFromDict":

    if not isinstance(pcd, str):
        cfg = cfg.copy()
        # set loading pipeline type
        cfg.data.test.pipeline[0].type = 'LoadPointsFromDict'

Then go to the file "loading.py" and see:

@PIPELINES.register_module()
class LoadPointsFromDict(LoadPointsFromFile):
    """Load Points From Dict."""

    def __call__(self, results):
        assert 'points' in results
        return results

So actually it still goes into the "class LoadPointsFromFile(object):"? But when I added a print to "class LoadPointsFromFile(object):", I found that the program execution did not enter the class.

[lwplw]

I solved it. I used the code below to actively process the input point cloud numpy array format data before sending it into the network, which passed and got the same result as the file path of the loaded .bin.

# numpy array to LiDARPoints
points_class = get_points_type(coord_type)
pointcloud_np = points_class(pointcloud_np, points_dim=pointcloud_np.shape[-1], attribute_dims=None)

Although I achieved the result I wanted, I am still not sure if there is a problem with the numpy array format I entered, or if there is a bug in the interface code of mmdetection3d.

[songhc8]

I solved it. I used the code below to actively process the input point cloud numpy array format data before sending it into the network, which passed and got the same result as the file path of the loaded .bin.

# numpy array to LiDARPoints
points_class = get_points_type(coord_type)
pointcloud_np = points_class(pointcloud_np, points_dim=pointcloud_np.shape[-1], attribute_dims=None)

Although I achieved the result I wanted, I am still not sure if there is a problem with the numpy array format I entered, or if there is a bug in the interface code of mmdetection3d.

it works without error msg. bug i am not sure if it can be used to inference images continuously. from mmdet3d.core.points import BasePoints, get_points_type points_class = get_points_type('LIDAR') pointcloud_np = points_class(a, points_dim=a.shape[-1], attribute_dims=None) where a is a numpy array. whose shape is Nx3 or Nx4 . i.e. each row is one point whose dimension is xyz or xyzi(xyz with intensity).

[rookierobot]

I solved it. I used the code below to actively process the input point cloud numpy array format data before sending it into the network, which passed and got the same result as the file path of the loaded .bin.

# numpy array to LiDARPoints
points_class = get_points_type(coord_type)
pointcloud_np = points_class(pointcloud_np, points_dim=pointcloud_np.shape[-1], attribute_dims=None)

Although I achieved the result I wanted, I am still not sure if there is a problem with the numpy array format I entered, or if there is a bug in the interface code of mmdetection3d.

Hi~ I read the point cloud data from rostopic and convert it to numpy format, and I followed your approach and my code snippet is as follows:

gen = pc2.read_points_list(data)  #  import sensor_msgs.point_cloud2 as pc2
pointcloud_np = np.array(list(gen));

print("Point Cloud Data:")
print(pointcloud_np)

points_class = get_points_type('DEPTH')
pointcloud_np = points_class(pointcloud_np, points_dim=pointcloud_np.shape[-1], attribute_dims=None)
result, data = inference_detector(model, pointcloud_np)

But I encountered the following error:

Point Cloud Data:
[[-2.64505148e+00 -1.56626070e+00  2.70029902e+00  1.16151864e-38]
 [-2.62720919e+00 -1.56563520e+00  2.67942047e+00  1.15229922e-38]
 [-2.61430359e+00 -1.56564093e+00  2.67943239e+00  1.15229922e-38]
 ...
 [ 2.50220442e+00  1.38325202e+00  4.85836077e+00  4.05623917e-39]
 [ 2.57229471e+00  1.38324618e+00  4.85825348e+00  4.14843480e-39]
 [ 2.64359498e+00  1.36904907e+00  4.80983162e+00  2.67366906e-39]]

[ERROR] [1705407006.262815]: bad callback: <function callback at 0x7efe85b891f0>
Traceback (most recent call last):
  File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "fcaf3d_river.py", line 87, in callback
    result, data = inference_detector(model, pointcloud_np)
  File "/home/river/3d/mmdet3d/apis/inference.py", line 140, in inference_detector
    data = test_pipeline(data)
  File "/home/river/3d/mmdet3d/datasets/pipelines/compose.py", line 49, in __call__
    data = t(data)
  File "/home/river/3d/mmdet3d/datasets/pipelines/test_time_aug.py", line 213, in __call__
    data = self.transforms(_results)
  File "/home/river/3d/mmdet3d/datasets/pipelines/compose.py", line 49, in __call__
    data = t(data)
  File "/home/river/3d/mmdet3d/datasets/pipelines/loading.py", line 328, in __call__
    assert points.attribute_dims is not None and \
AssertionError: Expect points have color attribute

May I ask what went wrong? Is my point cloud data in the wrong format?

[rookierobot]

And then I check the loading.py,

def __call__(self, results):
        """Call function to normalize color of points.

        Args:
            results (dict): Result dict containing point clouds data.

        Returns:
            dict: The result dict containing the normalized points.
                Updated key and value are described below.

                - points (:obj:`BasePoints`): Points after color normalization.
        """
        points = results['points']
        assert points.attribute_dims is not None and \
            'color' in points.attribute_dims.keys(), \
            'Expect points have color attribute'
        if self.color_mean is not None:
            points.color = points.color - \
                points.color.new_tensor(self.color_mean)
        points.color = points.color / 255.0
        results['points'] = points
        return results

it means that My pointcloud data should have color attribute?

[zhangye1111]

And then I check the loading.py,

def __call__(self, results):
        """Call function to normalize color of points.

        Args:
            results (dict): Result dict containing point clouds data.

        Returns:
            dict: The result dict containing the normalized points.
                Updated key and value are described below.

                - points (:obj:`BasePoints`): Points after color normalization.
        """
        points = results['points']
        assert points.attribute_dims is not None and \
            'color' in points.attribute_dims.keys(), \
            'Expect points have color attribute'
        if self.color_mean is not None:
            points.color = points.color - \
                points.color.new_tensor(self.color_mean)
        points.color = points.color / 255.0
        results['points'] = points
        return results

it means that My pointcloud data should have color attribute?

Do you know what the problem is, please? I'm having a similar problem to yours.