depth and RGB alignment

[IndiGleb]

Required Info
Camera Model	{ R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version	{ legacy / 2.<?>.<?> }
Language	{C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment	{Robot/Smartphone/VR/AR/others }

Issue Description

How does depth and RGB alignment work inside the align class ?

[MartyG-RealSense]

Hi @IndiGleb Alignment with the align_to instruction is governed by the RealSense SDK's Align Processing Block, which can make automatic adjustments for differences between sensors regarding field of view size and resolution.

https://unanancyowen.github.io/librealsense2_apireference/classrs2_1_1align.html#ae512ba538a7d60c9dc1eb24fb705c875

Alignment will be performed using SSSE3 or CUDA depending on whether CUDA graphics acceleration support is enabled in the RealSense SDK. The SDK's CUDA support requires an Nvidia graphics GPU and is typically used with RealSense cameras on Nvidia Jetson computing boards.

https://github.com/IntelRealSense/librealsense/blob/master/src/proc/processing-blocks-factory.cpp#L13-L30

[IndiGleb]

Hi @MartyG-RealSense i want to know how the alignment algorithm works in more detail.

[MartyG-RealSense]

There is limited information available about the internal mechanics of RealSense alignment other than the resources mentioned above. There is a little more information about the automatic adjustment of FOV size here:

https://github.com/IntelRealSense/librealsense/tree/master/examples/align#overview

[IndiGleb]

Has anyone tried to solve the alignment problem using the standard Opencv computer vision tools ?

[IndiGleb]

As I understand in the sdk of the camera alignment is done by triangulation, perhaps there is some way to solve this without resorting to such complex algorithms

[IndiGleb]

I think the alignment can be solved by a perspective transformation, and I can even pick up such coefficients, but I can't figure out how I can determine them in real time

[MartyG-RealSense]

If you require the camera intrinsics and extrinsics for a particular resolution then you can find these using the RealSense SDK tool 'rs-enumerate-devices' by launching it in calibration information mode with the instruction below:

rs-enumerate-devices - c

[MartyG-RealSense]

Hi @IndiGleb Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.