3D Reconstruction/Scan using the D455

[HasanTheSyrian]

Required Info
Camera Model	D400
Firmware Version	2.54.2
Operating System & Version	Windows 11
Platform	PC

Issue Description

I've been trying to export a .ply file that I can use in something like pyntcloud or MeshLab but it seems like the RealSense viewer only exports a single frame while in the 3D view.

I found rs-kinfu which seems to be a one-stop solution but it's in C++

I'm trying to achieve the highest quality/detail so if there are sources for that (post-processing, optimized camera settings, etc.) would be appreciated.

[MartyG-RealSense]

Hi @HasanTheSyrian As you have a RealSense D455 camera, if you are scanning a small object then the RealSense-compatible commercial software LIPScan 3D may suit your needs, as you can place an object on a rotating turntable and scan all sides of the object. LIPScan supports the .ply format.

https://www.lips-hci.com/lipscan-3d

---

An alternative method may be to capture a separate .ply in the RealSense Viewer from each side of the object and then stitch the individual .ply files together into a single one with the open-source tool CloudCompare, which includes a Python plugin in its Windows installer.

https://www.danielgm.net/cc/

https://www.cloudcompare.org/doc/wiki/index.php?title=Merge

An example of a command that I found to launch CloudCompare on Windows whilst merging a set of .ply files is below.

CloudCompare.exe -O input1.ply -O input2.ply -O input3.ply -C_EXPORT_FMT E57 -MERGE_CLOUDS

[MartyG-RealSense]

Regarding your comment on another issue about pyrealsense2 not being built for Arm: the pyrealsense2 wrapper can now be installed for Arm using the command pip install pyrealsense2 if the Python 3 version is Python 3.7, 3.8 or 3.9. This is possible because of the aarch64.whl file included in the list of pyrealsense2 pip packages.

[HasanTheSyrian]

Hi @HasanTheSyrian As you have a RealSense D455 camera, if you are scanning a small object then the RealSense-compatible commercial software LIPScan 3D may suit your needs, as you can place an object on a rotating turntable and scan all sides of the object. LIPScan supports the .ply format.

https://www.lips-hci.com/lipscan-3d

An alternative method may be to capture a separate .ply in the RealSense Viewer from each side of the object and then stitch the individual .ply files together into a single one with the open-source tool CloudCompare, which includes a Python plugin in its Windows installer.

https://www.danielgm.net/cc/

https://www.cloudcompare.org/doc/wiki/index.php?title=Merge

An example of a command that I found to launch CloudCompare on Windows whilst merging a set of .ply files is below.

CloudCompare.exe -O input1.ply -O input2.ply -O input3.ply -C_EXPORT_FMT E57 -MERGE_CLOUDS

Is it possible to use the record function in RealSense Viewer and convert the bag file to ply instead? It seems like that would be easier.

A turntable scenario is our exact use case (we will build our own out of a stepper motor and 3d printed parts :3)

Also, what about separating the scanned object from the rest of the environment?

[HasanTheSyrian]

Regarding your comment on another issue about pyrealsense2 not being built for Arm: the pyrealsense2 wrapper can now be installed for Arm using the command pip install pyrealsense2 if the Python 3 version is Python 3.7, 3.8 or 3.9. This is possible because of the aarch64.whl file included in the list of pyrealsense2 pip packages.

Ah I was looking for "arm" in the build list I forgot its referred to as "aarch64"

[MartyG-RealSense]

The RealSense SDK has a bag extraction tool called rs-convert which can extract bag file frames as .ply files, but it would create multiple individual ply files, so if you were going to have to stitch those individual .ply together then just taking several side-scans in the RealSense Viewer would be better.

A way to isolate the turntable object from the background environment may be to set a Threshold Filter post-processing filter in the 'Post-Processing' section of the Viewer's options side-panel. It enables you to set a maximum distance at which depth detail will be rendered, so the maximum threshold could be set just far enough to reach the far side of the turntable.

In the video of LIPScan 3D and RealSense at the link below, it limits the scan's detail only to the object on the turntable.

https://www.youtube.com/watch?v=N14Pi6z-MkE

[HasanTheSyrian]

Thanks for the help.

How does it know the order of the images, do I open (in the command) each .ply image in a clockwise order (or is that information available already)? Can I use a top/bottom view image when scanning something like a cup?

I found the CLI documentation but it seems like it doesn't carry the answers to my questions.

Maybe you can tell me where you got the example command from, I found it being mentioned here, though it's only in a forum post.

[MartyG-RealSense]

Intel are not the developer of CloudCompare so I do not have advice to offer about how its functions work, unfortunately.

Yes, that forum entry is where I obtained the example command from.

The links below have information about using the Merge function of CloudCompare by menu selection within CloudCompare (the first link) or by launch command (the second link).

http://www.cloudcompare.org/doc/wiki/index.php?title=Merge

http://www.cloudcompare.org/forum/viewtopic.php?t=2543

[HasanTheSyrian]

Hmm... considering all of this. Wouldn't it be easier to use rs-kinfu? From my understanding, I can just grab the camera and move it around the scene and it outputs a single ply file that I can then convert to a mesh/3d object; is that correct?

[MartyG-RealSense]

If you are able to use C++ instead of Python then in most cases rs-kinfu will be the optimal solution for scanning an object from all sides with a single camera and then outputting a single .ply file that can be imported into a 3D software tool for conversion to a solid mesh.

[HasanTheSyrian]

Okay, thanks for your patience and answers.