Get X,Y coordinates from X,Y,Z coordinates of realsense /OPEN3D COORDINATES IN REALSENSE

[BIGINCHI]

Required Info
Camera Model	{ D400 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win (10)
Kernel Version (Linux Only)	-
Platform	PC
SDK Version	{ 2 }
Language	{opencv/python}

Issue Description

im using open3D to get x,y,z coordinates of the segments im detecting in my cloud . After detecting im converting the X,y,z coordinates to realsense coordinate systems , using the cloud points

_def transform_point_from_realsenseopen3d(cluster): return([cluster[0][0],cluster[0][1], -cluster[0][2]])

are there better ways to convert my coordinates and how?

this snippet helps in converting open3d coordinates to realsense cloud coordinates now im using opencv_cloudpoint_viewer.py from the wrappers/python/examples in librealsense i want to convert realsense X,Y,Z coordinates to X,Y coordinates to view on a picture that for which im using

_cv2.imwrite('./out.png', colorimage) in the opencv_cloudpoint_viewer.py

how do i get proper X,Y coordinates, from X,Y,Z coordinates and mark them using opencv circle function

[MartyG-RealSense]

Hi @BIGINCHI If the XYZ values of a pointcloud have been calculated with the SDK instruction rs2_deproject_pixel_to_point like line 72 of opencv_pointcloud_viewer.py does then the XYZ points can be converted back to XY pixels with the instruction rs2_project_point_to_pixel. https://github.com/IntelRealSense/librealsense/issues/1231#issuecomment-368421888 has a Python example of rs2_project_point_to_pixel.

In regard to marking with circles using OpenCV, the only case on tis forum that I could find is https://github.com/IntelRealSense/librealsense/issues/6239#issuecomment-613996151 where the RealSense user in that case uses OpenCV's HoughCircles() function. Their script is at https://github.com/IntelRealSense/librealsense/issues/6239#issuecomment-614674313 and although it is a C++ script, the OpenCV sections of the code that contain the HoughCircles() function may provide some helpful ideas.

[BIGINCHI]

Thank You @MartyG-RealSense for the quick reply , yes im using deprojection and its working when i run my open3d program it looks like this

here you can see 3 bounding boxes i get the coordinates and display in realsense viewer and it looks like this

there is a big difference in points distance and scale , the coordinate system for both is different if open 3D is X y z realsense has to be x,z,-y i need help in converting open3d X,y,z coordinates to scale according to realsense environment.

this is how it looks if i import the points directly without manipulating the coordinates.

Thank you in Advance.

[MartyG-RealSense]

Which RealSense camera model are you using, please?

Most of the 400 Series models have a default Depth Unit Scale of 0.001. The D405 model has a default scale though of 0.01.

When RealSense data is used with a non-RealSense tool, the depth scale of the non-RealSense tool should be set to the same depth scale as the RealSense camera. Otherwise, there will be incorrect scaling such as a pointcloud being too small.

[BIGINCHI]

im using the D435i camera, sure ill try that. could you also share any references on how to play around with the scale and depth pointcloud of realsense. Thanks a Lot.

[MartyG-RealSense]

The default depth unit scale of a RealSense camera can be changed to a custom value with the RealSense SDK instruction rs.option.depth_units as demonstrated in Python code at https://github.com/IntelRealSense/librealsense/issues/1905#issuecomment-398367795

[BIGINCHI]

one last question, im saving the images of the realsense cloud point using

_cv2.imwrite('./out.png', colorimage)

what i want to include in the picture is an axes too which is within the cloud point

[MartyG-RealSense]

I researched your question but did not find a way to write axes onto an image when saving it, unfortunately.

[BIGINCHI]

I researched your question but did not find a way to write axes onto an image when saving it, unfortunately.

Thank you for the help. I have no further questions regarding this.

[MartyG-RealSense]

Thanks very much. As you have no further questions, I will close this case. Thanks again!