search

moetsi / Sensor-Stream-Pipe

Open Source Sensor Stream Server and Client for Digitizing Reality
MIT License
70 stars 11 forks source link

IR frame encoding with Zdepth #5

Closed stytim closed 4 years ago

stytim commented 4 years ago

I am wondering is there a reason not using the Zdepth library to encode IR frame as well since depth and IR are both 16 bit images?

Thanks.

adammpolak commented 4 years ago

@stytim working on getting back to you!

amourao commented 4 years ago

Hi @stytim, you are correct in both IR and depth frames being 16 bit images with unsigned integer pixels. But there is a major difference between IR and depth frames:

Zdepth is optimized for millimeter pixel data (in the range of 0-11840) due to the way quantization is applied to the results, and cannot encode the IR values above 11840. You can check Zdepth's quantization in more detail here https://github.com/catid/Zdepth/blob/master/include/zdepth.hpp#L158

SSP still allows you to choose Zdepth compression for IR data, but you'll lose all pixel values above 11840. If your goal is to stream IR data, I recommend sticking with the Nvidia encoder or FFmpeg.

Let me know if you have any further questions regarding SSP.

stytim commented 4 years ago

Hi @stytim, you are correct in both IR and depth frames being 16 bit images with unsigned integer pixels.

But there is a major difference between IR and depth frames:

  • depth pixels represent the millimeter, ranging from 0 to ~12000 mm (12 meters)

  • IR pixels represent the raw infrared values, ranging from 0 to 65535

Zdepth is optimized for millimeter pixel data (in the range of 0-11840) due to the way quantization is applied to the results, and cannot encode the IR values above 11840.

You can check Zdepth's quantization in more detail here https://github.com/catid/Zdepth/blob/master/include/zdepth.hpp#L158

SSP still allows you to choose Zdepth compression for IR data, but you'll lose all pixel values above 11840.

If your goal is to stream IR data, I recommend sticking with the Nvidia encoder or FFmpeg.

Let me know if you have any further questions regarding SSP.

Thanks for your explanation. I found that even if I remove the quantization function in Zdepth, there is still some issues and the performance is not as good as NvPipe.