Reproducing ssp_tester result

[jaykim305]

Hi, thank you for the great project. I want to reproduce the result of ssp_tester as presented here, and I tried testing with libx264 for color encoding as documented here. But the compression ratio and bitrate are poor compared to your result.

Could you provide the configuration file (especially what codec and options you used) and video to make the result? Here are my config file and the result.

general:
  host: "localhost"
  port: 9999
  frame_source:
    type: "kinect"
    parameters:
      stream_color_video: True
      stream_depth_video: True
      stream_ir_video: False
      streaming_color_format: "K4A_IMAGE_FORMAT_COLOR_BGRA32"
      streaming_color_resolution: "K4A_COLOR_RESOLUTION_720P"
      streaming_depth_mode: "K4A_DEPTH_MODE_NFOV_UNBINNED"
      wired_sync_mode: "K4A_WIRED_SYNC_MODE_STANDALONE"
      streaming_rate: "K4A_FRAMES_PER_SECOND_30"
      absoluteExposureValue: 0
video_encoder:
  0: #color
    type: "libav"
    codec_name: "libx264"
    pix_fmt: "yuv420p"
    bit_rate: 2000000
    options:
      preset: "veryfast"
      crf: "23"
  1: #depth
    type: "zdepth"
  2: #ir
    type: "null"
    codec_name: "NVPIPE_HEVC"
    input_format: "NVPIPE_UINT16"
    bit_rate: 40000

[2022-08-25 11:47:59.112] [critical] [statistics];[1], Depth
[2022-08-25 11:47:59.112] [critical]    [time];[1];16.633333333333333;seconds
[2022-08-25 11:47:59.112] [critical]    [original_size];[1];367906712;bytes
[2022-08-25 11:47:59.112] [critical]    [compressed_size];[1];25259648;bytes
[2022-08-25 11:47:59.112] [critical]    [original_bandwidth];[1];176.94912000000002;Mbps
[2022-08-25 11:47:59.112] [critical]    [compressed_bandwidth];[1];12.14892889779559;Mbps
[2022-08-25 11:47:59.112] [critical]    [compression ratio];[1];14.564998;x
[2022-08-25 11:47:59.112] [critical]    [latency];[1];4750005.5;ms
[2022-08-25 11:47:59.112] [critical]    [MSE];[1];0.974396345860123
[2022-08-25 11:47:59.112] [critical]    [MSE_4096];[1];0.8059916654054637
[2022-08-25 11:47:59.112] [critical] [statistics];[0], Color
[2022-08-25 11:47:59.112] [critical]    [time];[0];16.633333333333333;seconds
[2022-08-25 11:47:59.112] [critical]    [original_size];[0];1839517592;bytes
[2022-08-25 11:47:59.112] [critical]    [compressed_size];[0];40926298;bytes
[2022-08-25 11:47:59.112] [critical]    [original_bandwidth];[0];884.73792;Mbps
[2022-08-25 11:47:59.112] [critical]    [compressed_bandwidth];[0];19.68399102204409;Mbps
[2022-08-25 11:47:59.112] [critical]    [compression ratio];[0];44.947083;x
[2022-08-25 11:47:59.112] [critical]    [latency];[0];133647180;ms
[2022-08-25 11:47:59.112] [critical]    [PSNR];[0];40.380309822277255
[2022-08-25 11:47:59.112] [critical]    [MSSIM];[0];0.9722954843038729;0.9865859535347209;0.9778180558361117;0

P.S The libx264 doesn't seem to meet the target bitrate I specified in the config (2Mbps). I additionally set the codec parameters in the libav_encoder.cc as below: Is there any additional option that I have to consider ?

  av_codec_context_->width = width;
  av_codec_context_->height = height;
  /* frames per second */
  av_codec_context_->time_base.num = 1;
  av_codec_context_->time_base.den = (int)fps_;
  av_codec_context_->framerate.num = 1;
  av_codec_context_->framerate.den = (int)fps_;
  av_codec_context_->gop_size = 60;

  av_codec_context_->bit_rate_tolerance = 0;
  av_codec_context_->rc_max_rate = codec_parameters_["bit_rate"].as<int>();
  av_codec_context_->rc_buffer_size = codec_parameters_["bit_rate"].as<int>()*2;
  av_codec_context_->max_b_frames = 0;
  av_codec_context_->delay = 0;

[adammpolak]

What are platform are you building on?

[jaykim305]

I am using Ubuntu 18.04 with Azure Kinect DK. I know that the recommended Ubuntu is 20.04, but I managed to compile and run the binaries on 18.04 as well. Additionally, the pre-built ffmpeg av library provided in your CMake file below was not enabled with libx264 and I couldn't use the libx264 codec initially. https://github.com/moetsi/Sensor-Stream-Pipe/blob/297581ff7b050bc4e97c1861c1f3e4f0955a24b9/CMakeLists.txt#L81 I recompiled ffmpeg enabled with libx264 manually and now it works but cannot reproduce the result. My guess is that I'm missing some encoding parameters.

[adammpolak]

@jaykim305 great debugging there!

I think you are likely correct @jaykim305 let me check our records if I have the .yaml we used.

[adammpolak]

Sorry @jaykim305 unfortunately we do not have the .yaml file we used, sorry for the let down :(

[adammpolak]

@jaykim305

I believe I know the issue

I believe we have switched the project to use open264 rather than licensed version. https://github.com/AkillesAILimited/openh264

I believe this different "open" version performs differently.

[jaykim305]

Yes, I recall that the provided ffmpeg is built with disabling GPL and enabled with openh264 instead of libx264. I will switch to open264 and try it again. Thank you for the information!