rs2::pipeline can't playback two IR streams from bag file

[UnaNancyOwen]

Required Info
Camera Model	D400
SDK Version	2.10.3

Issue Description

rs2::pipeline can't correctly playback two IR streams from bag file that recorded two IR streams. Is this a specification of rs2::pipeline? Or is this a bug?

This code is record IR streams to bag file.

// Record IR Streams to File
rs2::config config;
config.enable_stream( rs2_stream::RS2_STREAM_INFRARED, 1, width, height, rs2_format::RS2_FORMAT_Y8, fps );
config.enable_stream( rs2_stream::RS2_STREAM_INFRARED, 2, width, height, rs2_format::RS2_FORMAT_Y8, fps );
config.enable_record_to_file( "two_ir_streams.bag" );

rs2::pipeline pipeline; 
const rs2::pipeline_profile pipeline_profile = pipeline.start( config );

// Output Enable Streams Name
const std::vector<rs2::stream_profile> stream_profiles = pipeline_profile.get_streams();
for( const rs2::stream_profile& stream_profile : stream_profiles ){
    std::cout << stream_profile.stream_name() << std::endl;
}

// e.g. Get 100 Frames for Record
for( uint32_t i = 0; i < 100; i++ ){
    rs2::frameset frameset = pipeline.wait_for_frames();
}

Output is following. It is correctly output the two IR streams. And, that recorded file can be playback two IR streams correctly using RealSense Viewer.

Infrared 1
Infrared 2

This code is playback IR streams from bag file that recorded two IR streams.

// Playback IR Streams from File
rs2::config config;
config.enable_device_from_file( "two_ir_streams.bag" );

rs2::pipeline pipeline; 
const rs2::pipeline_profile pipeline_profile = pipeline.start( config );

// Output Enable Streams Name
const std::vector<rs2::stream_profile> stream_profiles = pipeline_profile.get_streams();
for( const rs2::stream_profile& stream_profile : stream_profiles ){
    std::cout << stream_profile.stream_name() << std::endl;
}

Output is following. Infrared 2 is missing. Why?

Infrared 1

[zivsha]

I think this is a bug, try config.enable_all_streams and if the result is the same then this is a bug in config.h which the pipeline uses to resolve streams. You can work around it by iterating all streams and enabling each one separately.

[UnaNancyOwen]

@zivsha I added config.enable_all_streams() and tried again. But, the result didn't change. Infrared 2 is still missing.

  // Playback IR Streams from File
  rs2::config config;
  config.enable_device_from_file( "two_ir_streams.bag" );
+ config.enable_all_streams();

Infrared 1

[UnaNancyOwen]

I confirmed that this problem can avoid in following method. But, I think it is not smart method. I still think that this problem should be solved. I think that rs2::pipeline should be play all streams that contained when playback the bag file by default. @dorodnic What do you think?

// Retrieve Each Streams that contain in File
rs2::config config;
rs2::context context;
const rs2::playback playback = context.load_device( "two_ir_streams.bag" );
const std::vector<rs2::sensor> sensors = playback.query_sensors();
for( const rs2::sensor& sensor : sensors ){
    const std::vector<rs2::stream_profile> stream_profiles = sensor.get_stream_profiles();
    for( const rs2::stream_profile& stream_profile : stream_profiles ){
        config.enable_stream( stream_profile.stream_type(), stream_profile.stream_index() );
    }
}

// Playback IR Streams from File
rs2::pipeline pipeline;
config.enable_device_from_file( playback.file_name() );
const rs2::pipeline_profile pipeline_profile = pipeline.start( config );

// Output Enable Streams Name
const std::vector<rs2::stream_profile> stream_profiles = pipeline_profile.get_streams();
for( const rs2::stream_profile& stream_profile : stream_profiles ){
    std::cout << stream_profile.stream_name() << std::endl;
}

Infrared 1
Infrared 2

[dorodnic]

Hi @UnaNancyOwen Yes, your assumptions about the API are reasonable, we should fix it.

[dorodnic]

@UnaNancyOwen, the problem has been addressed in 2.14.0, along with significant refactoring of pipeline request resolution. In particular, enable_all / default start for recordings will always select all recorded streams. We hope the new solution will offer more consistent and predictable behavior. Thank you for reporting this issue. If you have any problem with the new solution please open new issue (and reference this one).