colored point cloud streaming

[bmegli]

Infrared (greyscale, D435) point cloud streaming works since previous release.

A proof-of-concept for colored point cloud streaming was implemented in RNHVE depth-color branch along #13.

The problem with colored streaming stems from the facts:

• infrared sensor is physically in different location than color sensor
• infrared sensors are used for stereo vision (depth)
• infrared sensor has different characteristics compared to color sensor (FOV, etc)

This means that there is alignment problem between depth and color data.

To get most from hardware for streaming it is important to consider:

• the direction of alignment (depth to color vs color to depth)
• how resolution affects FOV of sensors
• also in context of alignment (so downsampling/upsampling)
• cropping results in case of alignment of narrower to wider FOV
• how distortions affect alignment (e.g. camera distortion models)

Those problems hit:

• RNHVE - hardware configuration, distortions, alignment
• HDU - 2D to 3D unprojection with distortion model
• UNHVD and unhvd-native - hardware and unprojection configuration, rendering

[bmegli]

Looking at librealsense software-device example, librealsense may be used on both:

• sending side
• receiving side

To unproject/align the data.

Currently we:

• unproject in HDU (receiving side)
• align in RNHVE (sending side proof-of-concept)

[bmegli]

RNHVE part is finished and documented.

In view of the fact that librealsense doesn't use distortion models there is:

• nothing left to do in HDU
• only documentation update in UNHVD

[bmegli]

Finished.

The only thing not touched is:

• cropping results in case of alignment of narrower to wider FOV

This would only happen when aligning color to depth. Aligning depth to color seems to give better results so I am not going to consider it now.