IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.6k stars 4.83k forks source link

Multi-camera pointcloud-stitching #13280

Closed Havertz7K closed 2 months ago

Havertz7K commented 2 months ago

Required Info
Camera Model { R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only) (e.g. 4.14.13)
Platform PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version { legacy / 2.<?>.<?> }
Language {C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment {Robot/Smartphone/VR/AR/others }

Issue Description

I've tested pointcloud-stitching demo, and got a proper result. Now we need to stitch the views from six realsense D455f cameras, but now the demo only supports two. So is there any solutions to the problem? Thanks.

MartyG-RealSense commented 2 months ago

Hi @Havertz7K The section of code in pointcloud-stitching.cpp at the link below limits the number of cameras that can be used to 2. It may be worth changing '2' to '6' to see what happens.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/pointcloud/pointcloud-stitching/rs-pointcloud-stitching.cpp#L452-L456

In regard to other multicam pointcloud stitching solutions, the CONIX Center at Canegie Mellon created a C++ multi-camera pointcloud stitching system that could stitch up to twenty RealSense cameras.

https://github.com/conix-center/pointcloud_stitching

An Intel multicam demo in January 2018 took the approach of capturing with 4 separate PCs whose D435 cameras (one on each PC) were hardware synched, and sending the point cloud data to a 5th PC for combining and then doing post processing with the help of Unity.

https://www.intelrealsense.com/intel-realsense-volumetric-capture/

The link below describes how to use ROS1 to access three cameras on more than one PC. This method would likely be expandable to more than three cameras.

https://github.com/IntelRealSense/realsense-ros/wiki/showcase-of-using-3-cameras-in-2-machines

MartyG-RealSense commented 2 months ago

Hi @Havertz7K Do you require further assistance with this case, please? Thanks!

Havertz7K commented 2 months ago

No,thanks

-----原始邮件----- 发件人:MartyG-RealSense @.> 发送时间:2024-08-30 14:54:42 (星期五) 收件人: IntelRealSense/librealsense @.> 抄送: "Yufei Lei" @.>, Mention @.> 主题: Re: [IntelRealSense/librealsense] Multi-camera pointcloud-stitching (Issue #13280)

Hi @Havertz7K Do you require further assistance with this case, please? Thanks!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

MartyG-RealSense commented 2 months ago

Okay, thanks very much for the update!

zhanglepy commented 1 month ago

I have seen C++programs that achieve this goal, but I don't seem to have found a Python program that implements similar functionality. I look forward to your help.

MartyG-RealSense commented 1 month ago

@zhanglepy One method of pointcloud stitching for Python would be using an affine transform via the SDK instruction rs.rs2.transform_point_to_point to set multiple individual pointclouds to have the same position and rotation in 3D space - please see https://github.com/IntelRealSense/librealsense/issues/5583#issuecomment-570805578