D435 for 3D motion tracking

[J-mohajeri]

|---------------------------------|------------------------------------------- | | Camera Model | { D435 } | | Firmware Version | {05.09.02.00} | | Operating System & Version | {Win10} | | Kernel Version (Linux Only) | | | Platform | {Laptop AsusX450} | | SDK Version | { 2.16.5 } | | Language | {N.A} | | Segment | |

I have started to use the Intel RealSense SDK 2 technology for 3D motion recording, and would appreciate if you could help me with choosing an appropriate strategy. I used the device (D435) to record BAG files. The moving object was a mechanical device, shown bellow:

Now, I would like to generate a time history data of X, Y, Z movements of the object. Since it's a multi (rigid) body with two arms a main hinge, motion data of only 3 points are enough for my analysis.

Could you guide me with choosing a suitable code/software to use for my analysis? I already did my search, but it wasn't conclusive.

Thank you, Javad

[MartyG-RealSense]

If you want 3-dimensional XYZ values, the RealSense Viewer software has a 3D mode that can record 3D 'point cloud' data as a .ply format point cloud file instead of a bag. This ply file can then be imported into other software packages if necessary for further editing.

For example, the ply can be imported into the MeshLab software, where the point cloud data could be converted into a solid mesh.

https://www.andreasjakl.com/capturing-3d-point-cloud-intel-realsense-converting-mesh-meshlab/

You can access point cloud mode in the RealSense Viewer by left-clicking on the '3D' button in the top corner of the Viewer.

Point clouds can also be generated and exported as a ply file using scripting if you are creating your own application.

[J-mohajeri]

Thank you MartyG. I already used the PLY files to create a 3D model of the mechanical device. I used MeshLab and SolidWorks to do it, and the 3D model was created successfully.

My current aim (and my question in this post) is different from 3D scanning and creating 3D models. I would like to record 10 seconds of a dynamic process (movements of the mechanical device). Then, X, Y, Z values of the motion of the mechanical device over time can be created. Something similar to the below graph. This graph is created using 2D video tracking, but I would like to add in-depth (Z) values to it as well:

[MartyG-RealSense]

Apologies for the delay in responding. I am thinking that in this situation, putting marker images on the machine called ArUco markers might be appropriate. The camera can then detect these markers (which can be simple flat paper print-outs) and track position and pose information.

There was an unofficial user-created extension of RealSense's Unity engine wrapper last year that had a demonstration of ArUco marker image tracking with RealSense and Unity.

Using markers would make it easy to track exactly the points of the machinery that you want to follow.

The unofficial RealSense Unity wrapper extension can be found here:

https://github.com/ajingu/RealSense

It is quite out of date now compared to the current official Unity wrapper, though the possibility exists of grafting the ArUco system of this old wrapper into the current official one.

If Unity is not appropriate for your project, you can also track ArUco markers with ROS.

http://wiki.ros.org/aruco

[RealSenseCustomerSupport]

---

Hi J-mohajeri, Another option to consider would be D435i. This unit combines the D435 with an IMU unit. Please check out Slam with D435i example: https://github.com/intel-ros/realsense/wiki/SLAM-with-D435i