Request enhancements: rs-convert wants 3d distance csv support

[jianjieluo]

Issue Description

rs-convert in SDK version 2.12.0 is awesome!

I wonder whether it will support to convert each ROS-bag file to a format (maybe csv) that contains the real 3d distances info of each pixel in the depth matrix? #1868

It seems to use rs2_deproject_pixel_to_point(...) to get the distances in x and y axis like z axis using get_distance() or something like that.

It is my first time to new an issue for enhancements, and I will give reply if my statement makes you confused.

Thx.

[dorodnic]

Hi @longjj, thanks for the feedback. What would be the exact content of the CSV you would like to be added?

[jianjieluo]

Hi @dorodnic,

A CSV that records the x, y, z of each point in the 3D coordinate in the same unit is OK. Here are some contents formats I suggest.

Example 1

In the CSV, each line contains 3 value, representing x, y, z in 3D coordinate in the same unit of one pixel in the depth_frame. For example, I can use get_distance(i, j) to get the z (depth) in meters, then use rs2_deproject_pixel_to_point(...) to get the pixel (i, j) corresponding x and y in meters.

x1, y1, z1,
x2, y2, z2,
...

Example 2

Considering there are some invalid pixels in the depth_frame, where get_distance(i, j) returns 0, we can skip these invalid pixels and only store the valid points to reduce redundancy, while I need to record the (i, j) at this time for 3D reconstruction. That is, in the CSV, each line has 5 value to represent a point:

i1, j1, x1, y1, z1, 
i2, j2, x2, y2, z2,
...

or something like that.

---

In conclusion, I consider the feature for converting the pixels in the whole depth_frame to the points in the 3D coordinate is very useful and convenient to collect a large training data for the following deep learning object recognization research based on the depth information.

Since I am a green hand in structured light and 3D reconstruction, maybe I have missed some techs or functions in the Intel-realsense SDK2.0 which may have already solved this problem. Please remind me if it does exist, thanks a lot.

[dorodnic]

We can add this in a future release. Please note that what you are asking is very similar to the content of PLY exporter, the only difference being that PLY format we use is binary and not textual. However, it is essentially a binary dump of (x, y, z) coordinates.

[jianjieluo]

@dorodnic Thank you so much!

I used matlab to read the PLY format file. It's so cool !

Enlarge the red region and I can see my face!

I will try to use PLY file first, while I consider the textual representation of (x, y, z) is more intuitive and more concise if we don't care the rgb of an exact point.

In addition, should I close the issue now? I am not sure since it is my first time to new an issue for enhancements.

[jianjieluo]

What's more, I consider the example 2 above is preferred. Because with corresponding (i, j) information at the same time, we can easily select regions that we are interested in the color frame using the traditional digital image processing and then track the corresponding (x, y, z) in the 3D coordinate, which is very helpful in labeling training data for the researchers. And I think maybe the current version of ply converter doesn't have such a feature.

[dorodnic]

Keeping the issue open will help our team to track and integrate it into our internal backlog.

[RealSenseCustomerSupport]

---

Hello @longjj My name is Marty Grover and I will be handling your case.

Could you let me know whether you still require assistance with this case, please?

Thanks very much,

Marty Grover Intel RealSense Support Engineer

[MartyG-RealSense]

Update: the Intel feature request to export 3D depth information in a csv that was opened in June 2018 is still active but has not had progress for 2 years at the time of writing this as it is considered to be a low-priority feature.