search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.55k stars 4.81k forks source link

Issues with Frame Drop and Synchronization Using Multiple Intel RealSense D455F Cameras #13383

Open absargana opened 2 days ago

absargana commented 2 days ago

Hi everyone, I'm working on a project involving head pose estimation and eye blink detection using three Intel RealSense D455F cameras in a recording room. The goal is to capture synchronized RGB and depth streams. Here’s my setup: • Camera Model: D455F (3 cameras) • Firmware Version: 2.55.16.486 • Operating System: Windows 11 • SDK Version: 2.55.16.486 • Programming Language: Python • Recording Tool: RealSense Viewer • Connection: All cameras are connected via a USB hub to my computer. I'm saving the recordings as .bag files. However, I’m facing a couple of challenges:

  1. Frame Drop: When using all three cameras simultaneously, I experience up to a 25% frame drop, which significantly affects my data quality.
  2. Synchronization: How can I achieve proper synchronization across the three cameras to ensure aligned data streams?
  3. File Format: Is it better to continue saving the output in .bag files or save the RGB and depth streams directly for easier processing? Any suggestions or best practices on handling these issues would be greatly appreciated. Thank you!
MartyG-RealSense commented 1 day ago

Hi @absargana Thanks very much for your questions.

  1. What is the specification of the computer that you are using with the cameras, please? Each camera that is enabled simultaneously consumes more of the computer's resources. It is recommendable to use a computer with at least an Intel Core i5 processor or equivalent for handling three simultaneous cameras.

  2. Whilst depth can be synchronized between multiple cameras using hardware sync (joining the cameras together with wiring linking to their sync pins (under a small removable cap on top of the camera), there is ony one mode for depth + rgb synchronization with hardware sync. Called genlock hardware sync, the RGB sync did not work well though and genlock sync is no longer supported by Intel (though is still accessible in the Viewer and SDK).

It may be better simply to start all cameras simultaneously using Python code, like the multiple camera project at the link below.

https://github.com/ivomarvan/samples_and_experiments/tree/master/Multiple_realsense_cameras

  1. If you need to access the recorded data and interact with it in real-time then .bag files are the best choice. If you only need to play back the files to view them but not interact with the data then recording to a non-interactive video format such as .avi or .mpg should be sufficient.