IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.64k stars 4.83k forks source link

matlab pointcloud #13550

Open lrl-hub opened 10 hours ago

lrl-hub commented 10 hours ago
Required Info
Camera Model D455
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Win (11)
SDK Version 2.54.1
Language matlab

I'm trying to figure out if it's possible to obtain the point cloud of a specific object using MATLAB. In the MATLAB toolkit provided by the SDK, I can get the real-time point cloud of the camera using "pointcloud_example", but I can't obtain the complete point cloud of a specific object (for example, a sphere, the point cloud in the shape of a sphere). I'd like to ask if there is such a MATLAB program that can achieve this.

MartyG-RealSense commented 10 hours ago

Hi @lrl-hub If you want an object-shaped pointcloud that only has depth detail of the specific object then an easy way to do that may be to put a non-glossy plain black background such as a sheet of card behind / under the object. Then the camera should not be able to read depth detail from the black surface and it will appear on the pointcloud as an empty black area around the object.

lrl-hub commented 10 hours ago

OK, thank you. I've got a general understanding. I have another question. When using "pointcloud_example" in MATLAB, only the point cloud of the real-time camera can be obtained. For example, for a sphere, only the point cloud of one side of it can be obtained, and the point cloud of the invisible part cannot be obtained. Is there a solution to this?

MartyG-RealSense commented 10 hours ago

Obtaining a point cloud of a sphere would be like obtaining one of the human face. From the front you can capture the face and a bit of the ears but not the sides of the head because the camera cannot see that detail from the front-on, straight ahead position. To obtain a 360 degree pointcloud you would have to take at least 2 captures (one from each side of the object) and stitch them together.

The RealSense SDK provides a C++ pointcloud stitching example for doing this, using MATLAB to obtain calibration information and feed it into the stitching program.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/pointcloud/pointcloud-stitching

lrl-hub commented 9 hours ago

In the link you provided, it uses two cameras. I want to ask if a D455 depth camera can achieve it.

MartyG-RealSense commented 9 hours ago

If you can only use a single camera then you will likely have to export different views to multiple .ply pointcloud data files and then stitch them together in MATLAB using ICP registration, as described in the multiple-ply stitching discussion at the link below.

https://uk.mathworks.com/matlabcentral/answers/259918-merging-point-clouds-to-get-a-3-d-model