Connected multiple (10pcs) D455 cameras but throw error when starting 10th pipeline

[Dumnyezu]

---

Required Info
Camera Model	D455
Firmware Version	(RealSense Viewer 2.43.0.3013
Operating System & Version	Win (10)
Platform	PC (specified below)
SDK Version	2.43.0.3013
Language	python3.7 64-bit

Issue Description

PC configuration: HP Z4 G4 Workstation

• Processzor: Intel(R) Xeon(R) I7 W-2245 CPU @ 3.90GHz 3.91 GHz
• RAM: 64 GB
• 6 pcs USB 3.0 port (2 pcs USB 3.0 hub)
• Videócard: NVIDIA Quadro RTX 4000
• SSD: 250 GB
• HDD: 5TB
• PC power supply: 1000w
• Windows 10 Pro for Workstations
• 2x startech 4 port PCIe USB 3.0 card with 4 dedicated 5 Gbps channels and SATA power

Hi!

The setup: I use for a project 10 pieces of D455 cameras. The aim of the project is to take 3D pointcloud within 1-2 secunds for generating 3D model of a "scanned" object. I experienced, that with python wrapper this could not achieve, so thats why i started just collect data to bag file, and when the last camera is starting the streaming then I ask for the timestampe of the frames and save it (not yet implemented). So later i can find the specific time stamped frames and throw away the rest. From the selected frames later i can process them and finally get a merged ply file of the object.

The 2 pcs of startech PCI USB hadrware connectec via the PCIe slots, and power supply connected through SATA cables. These 2 USB cards accepts 2x4 pcs of USB 3.0 device. I connected 8 pcs of cameras to the cards. I connected the rest 2 pcs of camera to the PC's built-in 6 pcs USB3.0 port.

USB cables: 5 cameras are connected via USB3.0 A-male-female extension cable and an 1 m USBA-USBC cable. To sum it up this is 4 m of cables. (3m cable: lanberg brand, 1 m cable: conrad ) the rest of the cameras are connected via USB3.1 gen1 C-A 3m cable. (microconnect brand)

theoritically the PC can provide enough power to the cameras, 10 camera consumes ~35W.

When I run the code it starts pipeline one by one, but after it reaches the 10th pipeline I hear the windows "detatach USB device" sound and the program stops here, some time it just says in the 10th waiting_for_frames method, that frames didn't arrive within 5000 ms, but last time it dropped this message: Message=MFCreateDeviceSource(_device_attrs, &_source) returned: HResult 0xc00d3ea2: "the videorecord device has already not found."

I would be so grateful if someone who got similar experiences or from Intel crew could reply to my issue. Thanks you!

below u can find the code:

Programming environment: VS code 2019 python3.7 64bit interpreter

# First import the library
import pyrealsense2 as rs
import datetime as dt
import time
import logging

logging.basicConfig(format='%(asctime)-15s [%(name)s-%(process)02d] %(levelname)-7s: %(message)s',
                    level=logging.DEBUG,
                    handlers=[                   
                        logging.StreamHandler()
                    ])

camera10 ='151422254963'
camera9 = '215122253047'
camera8 = '215122251704'
camera7 = '215122253232'
camera6 = '215122253472'
camera5 = '215122253879'
camera4 = '151422253946' #'947522072605'
camera3 = '151422251866'  #'947122070868'
camera2 = '151422252570'  #'141722079463'
camera1 = '151422250393'  #'947522070642'

# Declare pointcloud object, for calculating pointclouds and texture mappings
pc = rs.pointcloud()
# We want the points object to be persistent so we can display the last cloud when a frame drops
points = rs.points()

 # Create save_to_ply object
start = dt.datetime.now()
timestamp = start.strftime("%y%m%d%H%M%S")
logging.info(" starting cam inicializing...")

# camera1
pipeline01 = rs.pipeline()
config01 = rs.config()
config01.enable_device(camera1)
config01.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera1" + timestamp+ ".bag")
config01.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)

# camera2
pipeline02 = rs.pipeline()
config02 = rs.config()
config02.enable_device(camera2)
config02.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera2" + timestamp+ ".bag")
config02.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera3
pipeline03 = rs.pipeline()
config03 = rs.config()
config03.enable_device(camera3)
config03.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera3" + timestamp+ ".bag")
config03.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera4
pipeline04 = rs.pipeline()
config04 = rs.config()
config04.enable_device(camera4)
config04.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera4" + timestamp+ ".bag")
config04.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera5
pipeline05 = rs.pipeline()
config05 = rs.config()
config05.enable_device(camera5)
config05.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera5" + timestamp+ ".bag")
config05.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera6
pipeline06 = rs.pipeline()
config06 = rs.config()
config06.enable_device(camera6)
config06.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera6" + timestamp+ ".bag")
config06.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera7
pipeline07 = rs.pipeline()
config07 = rs.config()
config07.enable_device(camera7)
config07.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera7" + timestamp+ ".bag")
config07.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera8
pipeline08 = rs.pipeline()
config08 = rs.config()
config08.enable_device(camera8)
config08.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera8" + timestamp+ ".bag")
config08.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera9
pipeline09 = rs.pipeline()
config09 = rs.config()
config09.enable_device(camera9)
config09.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera9" + timestamp+ ".bag")
config09.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
# camera10
pipeline10 = rs.pipeline()
config10 = rs.config()
config10.enable_device(camera10)
config10.enable_record_to_file("d:/Ortezis VKE projekt 3D tests/bag files/bag-camera10" + timestamp+ ".bag")
config10.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 5) # paraméterek: (mélység/RGBkép, felbontásX, felbontásY, fileforma, FPS)
logging.info(" cams initialized  ...")

# Start streaming from both cameras
pipeline01.start(config01)
logging.info(" pipelines1 started ...")
pipeline02.start(config02)
logging.info(" pipelines2 started ...")
pipeline03.start(config03)
logging.info(" pipelines3 started ...")
pipeline04.start(config04)
logging.info(" pipelines4 started ...")
pipeline05.start(config05)
logging.info(" pipelines5 started ...")
pipeline06.start(config06)
logging.info(" pipelines6 started ...")
pipeline07.start(config07)
logging.info(" pipelines7 started ...")
pipeline08.start(config08)
logging.info(" pipelines8 started ...")
pipeline09.start(config09)
logging.info(" pipelines9 started ...")
pipeline10.start(config10)
logging.info(" pipelines10 started ...")

time.sleep(5)

frame01 = pipeline01.wait_for_frames()
frame02 = pipeline02.wait_for_frames()
frame03 = pipeline03.wait_for_frames()
frame04 = pipeline04.wait_for_frames()
frame05 = pipeline05.wait_for_frames()
frame06 = pipeline06.wait_for_frames()
frame07 = pipeline07.wait_for_frames()
frame08 = pipeline08.wait_for_frames()
frame09 = pipeline09.wait_for_frames()
frame10 = pipeline10.wait_for_frames()

logging.info(" got frames, stopping processing...")

pipeline01.stop()
pipeline02.stop()
pipeline03.stop()
pipeline04.stop()
pipeline05.stop()
pipeline06.stop()
pipeline07.stop()
pipeline08.stop()
pipeline09.stop()
pipeline010.stop()
logging.info(" pipelines stoped ...")

[MartyG-RealSense]

Hi @Dumnyezu Each camera that is attached to the same computer will consume a portion of that computer's processing resources. Although you have a very powerful computer hardware specification, it is possible that the computer reaches a tipping point where it has enough resources for simultaneously processing 9 cameras at 1280x720 resolution and 5 FPS but not 10. If you were using a higher FPS than 5 then the program may have stopped even earlier than the 10th camera.

Recording to file will also place a burden on the computer's CPU.

If the object that is being scanned to file is not moving then less computing resources should be used if you stop each pipeline after its capture is taken and start the next pipeline in the sequence, until finally the 10th pipeline is stopped and you have all 10 files.

[Dumnyezu]

Dear @MartyG-RealSense, Thank you for your reply. I have to admit I am already very thankful to your replies to other issues, i have already learnt so much for this moment. Now I only tried the problematic camera which was made connection problems . Now I changed its USB cable to a shorter one (4m -> 3m) and now working. It is very strange for me, because one of the camera has 5m usb cable and no problem with it.

To answer to your last paragraph, the problem is , that the object is moving, because it is living, therfore there must be 10 frames which are captured in one time (within 1 sec). So in theoritically when the last pipeline is opened then i should wait for an external signal (some input from a button) when I want the frames to be captured. After the extarnal input gives a signal i can get the timestamp and close the pipelines.

I leave this issue still open, if there will be further issue with my setup.

[MartyG-RealSense]

Because RealSense cameras can transmit high volumes of data, it is important to have a high quality cable when using cable lengths greater than 2 meters, otherwise degraded performance and disconnections may occur. If a cable has lower quality than others in your setup or has internal damage (for example, due to being bent repeatedly) then you may have problems with the camera that is using that cable.

[Dumnyezu]

Dear @MartyG-RealSense

Maybe this question was already answeared, and I remember i read it somewhere, that theoritically the cameras not interfere with other D455 cameras laser projectors. Can you confirm this?

Another question, is there any option with the "enable_record_to_file(self: pyrealsense2.config, file_name: str)" method that only record the frames from the result of "wait_for_frames(self: pyrealsense2.pipeline, timeout_ms: int=5000)". So i dont have to collect data of around one minute until the last camera's pipeline started. This would make easier the post processing.

And a last question, is there any working method for calibrating cameras (10 cams)? How would work with 10 cameras?

[MartyG-RealSense]

Multiple units of 400 Series cameras such as D455 do not interfere with each other. They are also not interfered with by non-RealSense infrared devices but can cause interference for those non-RealSense devices in return.

Rather than risk a time-out by preventing new frames from arriving, how about using a 'for' instruction to skip a specific number of frames so that the stream is running but frames are not being captured, and then after the specified number of frames to skip has elapsed then the script will start capturing.

This method is commonly used to skip the first several frames when the pipeline is started in order to give auto-exposure time to settle down so that the first several frames are not incorrectly lit. A Python example of this is at https://github.com/IntelRealSense/librealsense/issues/9800#issuecomment-929651776

Intel have a $1500 USD OEM Target Calibration Board product that can calibrate multiple cameras. It is targeted at engineering departments and manufacturing facilities rather than individuals. The OEM version of the Dynamic Calibration software supplied with it can calibrate both extrinsics and intrinsics (the free public version only calibrates extrinsics) and calibration can be controlled remotely through a Linux server.

https://store.intelrealsense.com/buy-intel-realsense-d400-cameras-calibration-target.html

[MartyG-RealSense]

Hi @Dumnyezu Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.