D455 How to save 16UC1 format depth image?

gongyeted commented 4 years ago

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):
- Consider checking out SDK examples.
- Have you looked in our documentations?
- Is you question a frequently asked one?
- Try searching our GitHub Issues (open and closed) for a similar issue.
All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info
Camera Model	{ R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version	{ legacy / 2.<?>.<?> }
Language	{C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment	{Robot/Smartphone/VR/AR/others }

Issue Description

Hello everyone. I have a problem when saving depth and rgb images stream using D455. At first, I tried to save depth and rgb images in png format referring to the save-to-disk.cpp, but it turns out to be very slow with fps down to 5. Then I thought it maybe better using bag file. So I used cfg.enable_record_to_file("new.bag"); However, when using rs-convert to get images, the depth pngs are in 3 chennels with color, not 16UC1 which I want, but I didn't use colorize in my code. It seems that the rs-convert will enforce the depth to be colored. Can anyone help me solve this problem? Or is there anyway to directly save images without droping frames? My codes are shown below.

`int main(int argc, char * argv[]) try { // Declare RealSense pipeline, encapsulating the actual device and sensors rs2::pipeline pipe; rs2::config cfg; cfg.enable_record_to_file("new.bag");

cfg.enable_stream(RS2_STREAM_DEPTH);
cfg.enable_stream(RS2_STREAM_COLOR);

pipe.start(cfg);

rs2::align align_to_color(RS2_STREAM_COLOR);

rs2::log_to_console(RS2_LOG_SEVERITY_ERROR);
// Create a simple OpenGL window for rendering:
window app(1280, 720, "RealSense Save Depth With RGBs");

// Declare depth colorizer for pretty visualization of depth data
// rs2::colorizer color_map;
// Declare rates printer for showing streaming rates of the enabled streams.
rs2::rates_printer printer;

while (app) // Application still alive?
{
    // Using the align object, we block the application until a frameset is available
    rs2::frameset frameset = pipe.wait_for_frames();

    // Align all frames to color viewport
    frameset = align_to_color.process(frameset);

    // With the aligned frameset we proceed as usual
    auto depth = frameset.get_depth_frame();
    auto color = frameset.get_color_frame();
    // auto colorized_depth = c.colorize(depth);

    app.show(frameset);
}

return EXIT_SUCCESS;

}`

MartyG-RealSense commented 4 years ago

Hi @gongyeted There is an alternative bag to image extraction program for the 400 Series cameras called rs_bag2image that outputs monochrome depth images. It extracts depth frames as png, whilst it extracts rgb and infrared as jpg.

https://github.com/UnaNancyOwen/rs_bag2image/

gongyeted commented 4 years ago

Hi @gongyeted There is an alternative bag to image extraction program for the 400 Series cameras called rs_bag2image that outputs monochrome depth images. It extracts depth frames as png, whilst it extracts rgb and infrared as jpg.

https://github.com/UnaNancyOwen/rs_bag2image/

I realized an issue that I used frameset = align_to_color.process ( frameset ) to align depth to color , but this line only changes frameset, which I didn't save manually. I'm not sure whether the depth bag saved is aligned too. Will you be kind to help me ? Or is there any way to record aligned bag by realsense-viewer? I can't find any button in the UI to do this

MartyG-RealSense commented 4 years ago

Data saved to a bag file is not aligned. You can use scripting to extract the data from the bag and perform alignment on it. An example of this is Intel's Python tutorial distance_to_object:

https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/distance_to_object.ipynb

The RealSense Viewer does not have an align function built into it though.

gongyeted commented 4 years ago

Data saved to a bag file is not aligned. You can use scripting to extract the data from the bag and perform alignment on it. An example of this is Intel's Python tutorial distance_to_object:

https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/distance_to_object.ipynb

The RealSense Viewer does not have an align function built into it though.

Thank you for your reply. Another question: has the color image recorded by realsense viewer been rectified with the camera distortion parameters?

MartyG-RealSense commented 4 years ago

Most stream types accessible in the Viewer are rectified. The Y16 IR format is not rectified. This is because it is used primarily for camera calibration rather than general everyday use in applications. You can use Y16 IR but it only has a small number of supported resolutions and FPS speeds, up to a maximum of 25 FPS. Also, you cannot have the depth stream active when the Y16 IR mode is being used.

Y16 is also available on the RGB mode, though in a monochrome image.

MartyG-RealSense commented 4 years ago

Hi @gongyeted Do you require further assistance with this case, please? Thanks!

MartyG-RealSense commented 4 years ago

Case closed due to no further comments received.

IntelRealSense / librealsense

D455 How to save 16UC1 format depth image? #7254

Issue Description