Angle and Distance (or position data) for D435i

[MartyG-RealSense]

Hi @MyOtherNamesWereTaken The case https://github.com/IntelRealSense/librealsense/issues/4391 may provide helpful guidance if you have not seen it already. In that case, a RealSense user writes scripting for obtaining roll, pitch and yaw with Python at https://github.com/IntelRealSense/librealsense/issues/4391#issuecomment-510701377 and a RealSense team member advised them in https://github.com/IntelRealSense/librealsense/issues/4391#issuecomment-515894672 about how to improve their script.

[MyOtherNamesWereTaken]

Hello @MartyG-RealSense

Yes I've seen the code but im confused with this comment here https://github.com/IntelRealSense/librealsense/issues/4391#issuecomment-515894672

What to i swap the crossed out code pieces with?

[MartyG-RealSense]

It looks as though a suggestion for code to substitute into the crossed out section was not provided in that case.

Another Python case at https://github.com/IntelRealSense/librealsense/issues/3409#issuecomment-471673994 provides an alternative method where gyro and accel values are stored in gyro and accel numpy arrays as gyrox, gyroy and gyroz (gyro array), and accelx, accely and accelz (accel array). Those gyro and accel xyz values should then be able to be retrieved from the numpy array.

[MyOtherNamesWereTaken]

Those values are the rad/s values, i want to convert them into ° (without /s) for x/y and z.

[MyOtherNamesWereTaken]

Is there a better approach for visual odometry?

[MyOtherNamesWereTaken]

Hi @MyOtherNamesWereTaken The case #4391 may provide helpful guidance if you have not seen it already. In that case, a RealSense user writes scripting for obtaining roll, pitch and yaw with Python at #4391 (comment) and a RealSense team member advised them in #4391 (comment) about how to improve their script.

as a matter of fact i dont think this code works at all.

1. Rotating around the camera lense axis doesn't change x and z, therefore its the y axis in this code
2. Rotating around the length axis changes the z value => z axis in this code
3. Rotating around the missing side changes both x and z which means there's something wrong with the calculation

Or I am misunderstanding this whole thing.

[MartyG-RealSense]

I do not have any information to offer about the operational principles of that script, unfortunately.

You could consider using ROS and the RealSense ROS compatibility wrapper for odometry, as described in the links below.

https://shinkansan.github.io/2019-UGRP-DPoom/SLAM https://github.com/IntelRealSense/realsense-ros/issues/2190

I note though the statement made at the beginning of this case: "I was thinking of starting with getting the x,y,z coordinates of the camera and/or object and the rotation of the camera to itself".

---

It sounds as though you are aiming to calculate the pose (position and rotation) of the camera and of an observed object. If you wish to obtain the pose of an observed object then it is difficult to avoid using ROS as the majority of the RealSense-compatible object pose solutions make use of it. The official OpenCV documentation does provide a Python pose estimation tutorial though that makes use of an OpenCV SolvePNP algorithm to obtain pose.

https://docs.opencv.org/4.x/d7/d53/tutorial_py_pose.html

If ROS is an option for you then you could consider the DOPE method. Such a system uses an RGB image to recognize the object as a box and then calculates its 6 Degrees of Freedom (6DOF) pose. A couple of RealSense-compatible example projects of this type are in the links below.

https://github.com/pauloabelha/Deep_Object_Pose https://github.com/avinashsen707/AUBOi5-D435-ROS-DOPE

Another way to obtain the pose of an object with a 400 Series camera is to attach an ArUco image tag to the object.

https://github.com/Jphartogi/ipa_marker_detection

---

Calculating the pose of a D435i camera itself is more complex. It is straightforward to obtain the angles of the camera. To obtain its position in relation to a map though, a SLAM navigation solution such as the 2019-UGRP-DPoom one linked to above or another SLAM solution such as RTABMAP, ORB-SLAM or Kimera may be required.

---

If budget allows then you could consider pairing the D435i with a RealSense T265 Tracking Camera, which contains the same IMU component as the D435i but additionally has specialized navigation processing hardware and inbuilt 'pose stream' support for calculating the position of the T265 and objects observed by it that have a type of image tag called Apriltag attached to them. Bear in mind though that the T265, whilst available for purchase, is now a retired RealSense model.

Intel has a guide in the link below for using a 400 Series camera and T265 Tracking Camer together on a shared mount.

https://www.intelrealsense.com/depth-and-tracking-combined-get-started/

[MartyG-RealSense]

Hi @MyOtherNamesWereTaken Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.