rs-record-playback.cpp

[SunshineDou]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	{ R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version	{ legacy / 2.<?>.<?> }
Language	{C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment	{Robot/Smartphone/VR/AR/others }

Issue Description

Hello I want to use rs-record-playback.cpp to record a video, in default configuration, after recording, I get .bag files, and I use rs-convert.cpp to convert the file into color and depth images, but the depth image are colorful, that's to say, its color scheme is"Jet", visual preset is "dynamic". but I want to get "fixed White to Black" depth image. so I modified a few lines of the rs-record-playback.cpp as follows:

// Declare depth colorizer for pretty visualization of depth data rs2::colorizer color_map; color_map.set_option(RS2_OPTION_HISTOGRAM_EQUALIZATION_ENABLED, 1.f); color_map.set_option(RS2_OPTION_COLOR_SCHEME, 3.f); // Black to White

and then I start the rs-record-playback.cpp , the picture it showed is "White to Black" mode as follows but when I use rs-convert.cpp to convert the .bag file, the depth image is still in default configuration, its still colorful, in"jet"mode what can I do to get White to Black image form it?

[SunshineDou]

or How to convert "jet" mode depth image to "Black To White" depth image

Jet mode

Black to White mode Thank you very much!

[HippoEug]

I think one possible solution is that during playback of a .bag file, you convert it to black and white.

rs2::playback playback = device.as<rs2::playback>();
if (pipe->poll_for_frames(&frames)) // Check if new frames are ready
{
    color_map.set_option(RS2_OPTION_HISTOGRAM_EQUALIZATION_ENABLED, 1.f);
    color_map.set_option(RS2_OPTION_COLOR_SCHEME, 3.f); // Black to White
    depth = color_map.process(frames.get_depth_frame());
}

I just tried it out and it works. Let me know if it works for you

[SunshineDou]

Thank you very much for your replying. I have tried, but using rs-convert.cpp to convert, the output depth image is still colorful as follows:

[HippoEug]

What did you modify in rs-convert.cpp?

My code snippet was modified in rs-record-playback.cpp, and I added the black and white conversion during playback.

[SunshineDou]

I did't modify rs-convert.cpp. I just modified rs-record-playback.cpp as above, after modifying the rs-record-playback.cpp, I record a file a.bag, and I use original rs-convert.cpp to convert into png files, but the png depth file I get from the bag file is still colorful, in "jet"mode. do you mean that I need to modify the rs-convet.cpp? Thanks again for your generously help

[HippoEug]

Oh no no, you got to change it in the recording part, since the playback part is independent.

rs-convert.exe will convert whatever is recorded in rs-record.cpp, while rs-playback.cpp will display whatever it is recorded in rs-record.cpp. Hence changing rs-playback.cpp will not do anything.

I am not too sure where to change either in rs-record.cpp, will let you know if I find a way too.

[dorodnic]

Hi @SunshineDou @HippoEug is correct. Recorder is just saving the raw data, converting it to color/grayscale is a post-processing step. If you want this capability from rs-convert you need to modify it a bit, or just write a short python script that does what you want. Something like:

import pyrealsense2 as rs
import numpy as np
import matplotlib

p = rs.pipeline()
c = rs.config()
c.enabe_device_from_file("filename.bag")
prof = p.start(c)
dev = prof.get_device()
playback = dev.as_playback_device()
playback.set_real_time(false); # to avoid dropping frames
clr = rs.colorizer()
clr.set_option(rs.option.color_scheme, 3.0)
while True:
     fs = p.wait_for_frames()
     df = fs.get_depth_frame()
     colorized = clr.colorize(df)
     data = colorized.get_data()
     array = np.asanyarray(data)
     matplotlib.image.imsave('name.png', array)

[HippoEug]

I also rewrote it on C++, modified from converter-png.hpp. You can find it under header files.

Please see line 38 onwards, in converter-png.hpp

if (frame && (_streamType == rs2_stream::RS2_STREAM_ANY || frame.get_profile().stream_type() == _streamType)) {
    if (frames_map_get_and_set(frame.get_profile().stream_type(), frame.get_frame_number())) {
        continue;
    }

    // HERE
    if (frame.get_profile().stream_type() == rs2_stream::RS2_STREAM_DEPTH) {
        // MODIFIED START 30112018
        _colorizer.set_option(RS2_OPTION_HISTOGRAM_EQUALIZATION_ENABLED, 1.f);
        _colorizer.set_option(RS2_OPTION_COLOR_SCHEME, 3.f);
        // MODIFIED END 30112018

        frame = _colorizer.process(frame);
    }

    std::stringstream filename;
    filename << _filePath
        << "_" << frame.get_profile().stream_name()
        << "_" << frame.get_frame_number()
        << ".png";

    std::string filenameS = filename.str();

    add_sub_worker(
        [filenameS, frame] {
        stbi_write_png(
            filenameS.c_str()
            , frame.get_width()
            , frame.get_height()
            , frame.get_bytes_per_pixel()
            , frame.get_data()
            , frame.get_stride_in_bytes()
        );
    });
}

[SunshineDou]

@dorodnic I have tried on ubuntu 16 , I installed the pyrealsense, and type the command as follows: python test3.py, the result is as follows: Traceback (most recent call last): File "test3.py", line 7, in rs.config().enabe_device_from_file(config,"a.bag") AttributeError: 'pyrealsense2.pyrealsense2.config' object has no attribute 'enabe_device_from_file' what's the problem? Thank you for your help~

[SunshineDou]

@HippoEug I have modified the converter-png.hpp and recompiled the rs-convert.cpp. but the output depth image is as follows; It's very weird that it has only about 30kB, what's the problem? have you tried it? do you get normal"Black to White" depth image?

[SunshineDou]

@dorodnic I have found that it missed "l" in line"c.enable_device_from_file("a.bag")" I add it and run again, but it still failed: Traceback (most recent call last): File "test3.py", line 10, in playback = dev.as_playback_device() AttributeError: 'pyrealsense2.pyrealsense2.device' object has no attribute 'as_playback_device'

[dorodnic]

Hi @SunshineDou - sorry, not at the office so can't test the script, just writing from memory. There might be some typos. I think its just .as_playback(). There rest should be Ok, but I'm not 100% sure on saving numpy array to PNG (the last line). If this doesn't work, I'd try cv2.imwrite('file.png',array) (and pip install opencv-python)

[HippoEug]

Unfortunately I won't be able to use the camera until monday. I ran the code and everything seemed normal.. not sure about how big the file was either!

[SunshineDou]

@dorodnic Thanks for your help sincerely. as I am not familiar with this and I haven been confused by it during these time. I have tried and found that saving numpy array to PNG (the last line) doesn't work. so I tried cv2.imwrite('file.png',array) and it worked. the code as follows: `import pyrealsense2 as rs import numpy as np import matplotlib import cv2

p = rs.pipeline() c = rs.config() c.enable_device_from_file("test.bag") prof = p.start(c) dev = prof.get_device() playback = dev.as_playback() playback.set_real_time(false); # to avoid dropping frames clr = rs.colorizer() clr.set_option(rs.option.color_scheme, 3.0) i=1 while True: fs = p.wait_for_frames() df = fs.get_depth_frame()

cf = fs.get_color_frame()

 colorized = clr.colorize(df)
 data = colorized.get_data()
 array = np.asanyarray(data)
 cv2.imwrite(str(i)+'.png',array)
 i = i+1`

after I run it, the result as follows: File "test3.py", line 13, in playback.set_real_time(false); # to avoid dropping frames NameError: name 'false' is not defined

what's the problem? how to modify it? if I comment the line "playback.set_real_time(false); # to avoid dropping frames", it can work,the depth image I get are about 200KB, but it missed frames. and as I want to get all the frames in it, so I add a variable "i", but it cannot stop, it generates images constantly. Thanks again for your generously help

[SunshineDou]

@HippoEug Thank you very much for your help could you please have a try some day later? as I have tried but the output is very strange

[SunshineDou]

@HippoEug I have another question is that the depth image I get is three channels 24bit image, as I want to get one channel 16bit depth image and its corresponding color image to make datasets for the SLAM"Elasticfusion". so after I get the depth image from above python code, I use matlab code to convert it into one channel 16 bit depth image, the code is as follows: %RGB Depth to 16bit Depth script for i = 1:130 I = imread(['data/depth/',num2str(i),'.png']);%RGB 的深度图 R = I(1:480,1:640,1:1);%取一个通道 R_16 = im2uint16(R); imwrite(R_16,['data/depth16bit/',num2str(i),'.png']) end this is the image before convert this is the image after convert

I want to ask if I use the matlab code to convet it, is it the same as the original raw depth image? can I do in this way?or it has lossed accuracy?

[HippoEug]

I did the converter-png modification using 1280x720 resolution.

Before modification:

[556kb]

After modification:

[351kb]

[SunshineDou]

@HippoEug Thank you very much for your help, I recompiled it again using 640x480 resolution. the output depth image is about 120KB as follows:

[HippoEug]

@SunshineDou That seems like a good sign, the math adds up. 640x480 is 1/3 of 1280x720, so your depth image size is approximately 1/3 as well

[SunshineDou]

@HippoEug yeah, thank you very much for your help sincerely. but i found a weird phenomenon, that the converted depth image's number is smaller than color images, as follows:

[HippoEug]

Yeah that happens to me as well. All the frames skipping.. but for me it is not a problem, since I only needed 1 frame instead of every single frame captured.