IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.6k stars 4.83k forks source link

Doubts in using align #8344

Closed AJMena closed 3 years ago

AJMena commented 3 years ago

Required Info
Camera Model { D400 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Win 10
Kernel Version (Linux Only) (e.g. 4.14.13)
Platform PC
SDK Version {2.41.0.2657 }
Language {C++ }
Segment {others }

Issue Description

We are trying to measure the distances in 3D between some objects. We calculate the 3D coordinates of the points, export them, and then measure in CloudCompare (an external program). We calculate the 3D coordinates like its described in this article: https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0#processing-blocks-helpers We are wondering if we must align the frames to depth, to color or not align before calculating the points. The aligned process is described in the article too. We did some tests. We exported the point cloud without using align (default), using align to depth and using align to color. The results are slightly different. The color of the points is moved between the default and align to depth but the points are in the same coordinates and in align to depth some points do not have color. The points are translated between the default and align to color result.

So, how should we do it?

MartyG-RealSense commented 3 years ago

Hi @AJMena There are some pre-made options available for measuring between two points. It can be done with the Measure option in the 3D point cloud mode of the RealSense Viewer program (dragging a line with the mouse between two points on the point cloud), or with the SDK's rs-measure C++ example program.

https://github.com/IntelRealSense/librealsense/tree/master/examples/measure

rs-measure takes the approach of aligning color to depth instead of the most commonly used alignment method of depth to color. Developing a RealSense measuring application based on rs-measure may enable you to not have to use an external solution such as CloudCompare.

I have a recollection that one of the reasons for choosing color to depth in that example program though was simply to demonstrate how to perform color to depth alignment in the SDK, and a Python conversion of rs-measure by a RealSense user used depth to color alignment instead. So the difference between choosing one method or the other may be minimal, as your own tests suggested.

The alignment process aligns the data to the field of view (FOV) size of the source that you are aligning to. For example, depth to color alignment aligns the depth to the color FOV size. On the D435 and D435i models, the color FOV is smaller than the depth FOV, which may result in depth coordinates on the outer regions of the image being excluded from the image after alignment. The rs-align example explains these principles in its opening paragraph.

https://github.com/IntelRealSense/librealsense/tree/master/examples/align

If you are using the Windows version of the SDK, a pre-built executable version of rs-measure that can be launched with a double-leftclick can be found in the SDK's Tools folder. An easy way to find this folder is to right-click on the RealSense Viewer's launch sortcut icon on the Windows desktop and select the Open file location menu option.

AJMena commented 3 years ago

Hello MartyG, thanks for the quick response. I have already checked all that information before and test the demos. But I still have doubts. In this example the function "calculate" is used to obtain the coordinates of the points: https://github.com/IntelRealSense/librealsense/tree/master/examples/pointcloud But there is not any align done before. So, is it correct? Maybe that is omitted for simplification.

MartyG-RealSense commented 3 years ago

There are different approaches that can be used for point cloud generation. A couple of the most popular are detailed below.

Using pc.calculate could be considered the most quick and simple way of generating a point cloud. If you just require a point cloud and do not need to do further processing with the data then this may be an appropriate option.

Alternatively, a point cloud can also be generated using alignment and also deprojecting the pixels with the instruction rs2_deproject_pixel_to_point (essentially converting 2D pixels into 3D points). This approach may be used by a developer if a program needs to retrieve the XYZ values of coordinates.

The Projection documentation describes rs2_deproject_pixel_to_point with the quote below:


Deprojection takes a 2D pixel location on a stream's images, as well as a depth, specified in meters, and maps it to a 3D point location within the stream's associated 3D coordinate space. It is provided by the header-only function rs2_deproject_pixel_to_point

AJMena commented 3 years ago

Understood. So, for the moment we will use only pc.calculate. Thank you for your help.

MartyG-RealSense commented 3 years ago

You are very welcome. Good luck!