Print the matrix of depth image of one frame

[TED-EE]

---

Required Info
Camera Model	D415
Firmware Version	05.12.03.00
Operating System & Version	Win10
Kernel Version (Linux Only)	N/A
Platform	PC
SDK Version	2.0
Language	C++
Segment	others

Issue Description

I would like to print the matrix of the depth image of one frame for some image processing work. With Python, I made it work.

`

def depth_image():
np.set_printoptions(threshold=np.inf)
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
pipeline.start(config)

for i in range(100):

    frames = pipeline.wait_for_frames()

    depth_frame = frames.get_depth_frame()

depth_image = np.asanyarray(depth_frame.get_data())

colorizer = rs.colorizer()
colorizer.set_option(rs.option.color_scheme, 2)
colorized_depth = np.asanyarray(colorizer.colorize(depth_frame).get_data())
colorized_depth = cv2.cvtColor(colorized_depth, cv2.COLOR_BGR2GRAY)

cv2.imshow('1', colorized_depth)
print(colorized_depth)

key = cv2.waitKey(1000)

if key & 0xFF == ord('q') or key == 27:
    cv2.destroyAllWindows()
return colorized_depth

`

However, when it came to C++, I could not do it. When I modified the while (waitKey(10000) && cvGetWindowHandle(depth_win)) to for(int i = 0; i<100; i++) loop just as written in Python, it did not work.

`

int getDepth() try {

rs2::colorizer color_map;

color_map.set_option(RS2_OPTION_COLOR_SCHEME, 2); // white to black

rs2::pipeline pipe;

rs2::config pipe_config;
pipe_config.enable_stream(RS2_STREAM_DEPTH, 640, 480, RS2_FORMAT_Z16, 30);
pipe_config.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_BGR8, 30);

rs2::pipeline_profile profile = pipe.start(pipe_config);
const char* depth_win = "depth_Image";
namedWindow(depth_win, WINDOW_AUTOSIZE);
//const char* color_win = "color_Image";
//namedWindow(color_win, WINDOW_AUTOSIZE);

//    float depth_scale = get_depth_scale(profile.get_device());
//    rs2_stream align_to = find_stream_to_align(profile.get_streams());

while (waitKey(10000) && cvGetWindowHandle(depth_win)) {
//for (int i = 0; i < 100; ++i) {
    rs2::frameset data = pipe.wait_for_frames();

    rs2::frame depth = data.get_depth_frame().apply_filter(color_map);
    //rs2::frame color = data.get_color_frame();

    const int depth_w = depth.as<rs2::video_frame>().get_width();
    const int depth_h = depth.as<rs2::video_frame>().get_height();
    //const int color_w = color.as<rs2::video_frame>().get_width();
    //const int color_h = color.as<rs2::video_frame>().get_height();

    Mat depth_image(Size(depth_w, depth_h), CV_8UC3, (void*)depth.get_data(), Mat::AUTO_STEP);
    /*Mat color_image(Size(color_w, color_h), CV_8UC3, (void*)color.get_data(), Mat::AUTO_STEP);*/

    imshow(depth_win, depth_image);
    cout << depth_image << endl;
    /*imshow(color_win, color_image);*/
}
return EXIT_SUCCESS;

} catch (const rs2::error& e) { std::cout << "RealSense error calling" << e.get_failed_function() << "(" << e.get_failed_args() << "):\n" << e.what() << endl; return EXIT_FAILURE; } catch (const std::exception& e) { std::cout << e.what() << endl; return EXIT_FAILURE; }

`

Thanks.

[MartyG-RealSense]

Hi @Ted-EE May I clarify please if you mean that you want to export a depth matrix of an image?

If so, a way to do this may be to dump the data to a binary file format such as csv or bin. The subject is discussed in the link below.

https://github.com/IntelRealSense/librealsense/issues/3491

Within that discussion, @RealSenseCustomerSupport offers a script for doing so.

https://github.com/IntelRealSense/librealsense/issues/3491#issuecomment-475389906

[TED-EE]

@MartyG-RealSense Thanks for your replying. I will look into this link.

[TED-EE]

@MartyG-RealSense I don't think that link can solve my problem. I would like to make the problem clear. Because I want to do some image processing work on the depth image captured by realsense in Visual Studio C++, it will be better if the depth image is output in matrix form so that I can find the row and the column of the matrix, I can do kind of image processing work like undersampling and Gaussian kernel processing on the matrix of the depth image.

[MartyG-RealSense]

Apologies for the delay in responding further, as I was considering and researching your case.

Would using pixels_buffer to generate an nxm matrix of pixels from the raw depth data in Z16 format meet your needs?

https://github.com/IntelRealSense/librealsense/issues/4026#issuecomment-494855742

[TED-EE]

Thanks for your replying, I will have a try.

[TED-EE]

Thanks for your help. It really helps.

[TED-EE]

Another question... Since I have included <opencv2/imgproc/imgproc.hpp>, <opencv2/core/core.hpp>, <opencv2/highgui/highgui.hpp>, <librealsense2/rs.hpp>

When I did like auto depth_mat = depth_frame_to_meters(pipe, depth_frame); I got error: C++ identifier "depth_frame_to_meters" is undefined. Also,

float distanceTocam = depth_frame.get_distance(x,y); Error: C++ class has no member "get_distance".

I was confused about that.

[MartyG-RealSense]

I'm not sure about this one. Though if you are accessing OpenCV, you may find the OpenCV-using depth example in the link below helpful as a reference. For example, the developer of that program includes realsense.h

https://github.com/UnaNancyOwen/RealSense2Sample/tree/master/sample/Depth

[TED-EE]

The version of my opencv is 3.0.0 and my OS is x86 because another library included must be based on x86. I am not sure if the version of opencv is too low. Because the function depth_frame_to_meters included in cv-helpers.hpp said "RealSense examples have been desinged and tested with OpenCV 3.4, Working with latest OpenCV 4 requires minor code changes". https://github.com/IntelRealSense/librealsense/tree/c3c758d18c585a237bb5b635927797aa69996391/wrappers/opencv

[MartyG-RealSense]

The release-notes documentation for OpenCV 3.4.9 says that the Windows installer (link below) is 32-bit, and that 64-bit users of that version need to compile OpenCV from source code.

https://sourceforge.net/projects/opencvlibrary/files/3.4.9/opencv-3.4.9-vc14_vc15.exe/download

[MartyG-RealSense]

Hi @Ted-EE Do you require further assistance with this case please?

[MartyG-RealSense]

Case closed due to no further comments received.