Multiple L515 time synchronization by ros

[jiejie567]

I want to use 4 L515 cameras simultaneously. However, the note in https://dev.intelrealsense.com/docs/lidar-camera-l515-multi-camera-setup does not mean trigger all the cameras at the same time. if I use this method to trigger them at the same time, the camera will not perform at 30 Hz. What should I do in detail? I'll be appreciated for your futrue reply.

[MartyG-RealSense]

Hi @jiejie567 Adding additional RealSense cameras to the same computer normally increases the processing demands placed on that computer as it consumes computing resources to handle those multiple cameras. For 400 Series cameras, a computer with a modern CPU that is rated i7 or better is recommendable for using 4 cameras simultaneously on the same computer.

In the case of L515, you can use its hardware sync system described in the quoted white-paper document to 'stagger' them so that they are not all active at the same time, therefore reducing processing burden. If the four L515 cameras are operated without this staggering then they would likely have the same need for a strong computer hardware specification as the 400 Series cameras.

[jiejie567]

Thank you for your soon reply. What I need is How should I set up in the ros launch file as well as the hardware setup. I don't want to 'stagger' 4 L515 cameras，I want them active at the same time.

[MartyG-RealSense]

If your computer has a strong enough hardware specification to handle four active cameras simultaneously then there are two main methods that can be used to launch 4 cameras simultaneously in ROS1 (Kinetic, Melodic, Noetic). You can use the rs_multiple_devices.launch launch file and specify a serial number for each camera, or alternatively use rs_camera.launch and launch each camera in its own separate ROS terminal. This is described at the link below.

https://github.com/IntelRealSense/realsense-ros#work-with-multiple-cameras

If you are using the ROS2 ros2_beta wrapper then the wrapper has a multiple camera launch file called rs_multi_camera_launch.py

[jiejie567]

Thank you for your quick reply. rs_multiple_devices.launch Cannot synchronize time among cameras as well as the timestamp. I want to get the same time stamp for 4 cameas at the same time. I only find white-paper for synchronizationof D400 cameras. I wonder how can I synchronize 4 L515 cameras.

[MartyG-RealSense]

You can set the Inter Cam Sync Mode value of a camera in the RealSense ROS wrapper using the inter_cam_sync_mode parameter. For L515, where an external trigger signal is used, all cameras would have inter_cam_sync_mode set to slave mode.

[jiejie567]

I think it is a good idea. Whether there are examples about the external trigger signal setting? I think the example in https://dev.intelrealsense.com/docs/lidar-camera-l515-multi-camera-setup cannot perform at 30 Hz. What's more, whether there are specific setting instructions about the inter_cam_sync_mode parameter.

[MartyG-RealSense]

On the L515 cameras, the Inter Cam Sync Mode value for slave is '1'. The multiple cameras should be able to perform at 30 Hz when using hardware sync. If hardware sync is not used then the performance of multiple cameras will be partly dependent on the computer's hardware and whether it can handle the processing demand.

It is also worth mentioning that sometimes there may be a small drop in FPS when accessing cameras at a defined speed of 30 FPS through the ROS wrapper compared to a full 30 FPS when accessing them in the RealSense Viewer tool.

You could set inter_cam_sync_mode from the launch instruction by adding inter_cam_sync_mode:=1 or define it in a json camera configuration file and load the json from your ROS launch instruction or launch file.

The L515 sync white paper is a very good guide regarding setting up the electronics of an external trigger mechanism.

[jiejie567]

In the image, 160ms is consumed for three frames, so I think it will not perform at 30Hz if using this mothod of the white paper.

[MartyG-RealSense]

I believe that the above chart is not related to whether the camera can run at 30 Hz, but the time at which a particular camera activates and takes its capture before deactivating so that another camera can then activate and take its capture.

This sequence of activate-deactivate will only occur if hardware sync is being used.

[jiejie567]

Because of the additional overhead 60ms(setup), I think it won't run at 30 Hz. If not, how should I set the high and low voltage to control the camera.

[MartyG-RealSense]

I would recommend making the assumption that it is acheiving 30 Hz for all cameras and it is doing so by offsetting the times at which each individual camera makes a capture. So even if each camera initiates capture at a different time, it is still capturing 30 Hz (or 30 frames per second) worth of frames, assuming that once the 4th camera in the sequence deactivates then it loops back to activating the 1st camera and begins the sequence again.

[jiejie567]

I‘ m sorry for my poor understanding ability. Every time for trigger, the setup comsumes 60ms. As a result, the total time for 30 frames (10 tiggers） will comsume (60+333)10>1000ms.

[MartyG-RealSense]

If the cameras are activated in sets of two like in the above diagram (which has 8 cameras) and it took 640 ms in that diagram to complete the sequence for 8 cameras then I would assume that it would be about half that tim in ms for 4 cameras if they were activated in two pairs of 2.

My knowledge of the mathematics of syncing is admittedly limited though, so I thank you for your patience.

[jiejie567]

Thank you for your patient answer. In this way, are the time stamps of 4 cameras same in ROS?

[MartyG-RealSense]

Hardware sync for L515 is not the same as hardware sync for 400 Series cameras. The 400 Series hardware sync system synchronizes the slave cameras' timestamps with a master trigger signal. With the L515 system though, it is not about timestamp sync but instead preventing multiple L515 cameras from interfering with each other.

I would therefore assume that the frame timestamps of each L515 camera will be the same as though hardware sync was not being used.

[jiejie567]

You mean that the timestamps are same even if I don't perform hardware synchronization?

[MartyG-RealSense]

That would be my expectation that the timestamps of each individual camera would not be synced to the trigger.

[jiejie567]

Thank you again for your patient reply. I know what you mean now. My 4 L515 cameras don't have overlapping FoVs, so the cameras cannot interfer with each other. What I want is making their timestamps same, so what should I do?

[MartyG-RealSense]

Enabling a feature called Global Time should help to synchronize multiple cameras on the same computer by providing a common timestamp, as described at https://github.com/IntelRealSense/librealsense/pull/3909

Global Time is enabled by default on 400 Series cameras and disabled by default on L515 cameras.

In https://github.com/IntelRealSense/realsense-ros/issues/796 though the developer of the global time feature, who also created the RealSense ROS wrapper, explains that enabling global time makes less difference in ROS than it does in librealsense because ROS already compensates for drift between timestamps.

[jiejie567]

Enabling a feature called Global Time should help to synchronize multiple cameras on the same computer by providing a common timestamp, as described at IntelRealSense/librealsense#3909

Global Time is enabled by default on 400 Series cameras and disabled by default on L515 cameras.

In #796 though the developer of the global time feature, who also created the RealSense ROS wrapper, explains that enabling global time makes less difference in ROS than it does in librealsense because ROS already compensates for drift between timestamps.

So what should we do to make the cameras' timestamp same ?

[MartyG-RealSense]

Please try adding global_time_enabled:=true to your roslaunch instruction. For example:

roslaunch realsense2_camera rs_camera.launch global_time_enabled:=true

[jiejie567]

I try this. I can find that they all have used the global time. However, their timestamps are not same.

[MartyG-RealSense]

https://github.com/IntelRealSense/librealsense/issues/10189 is a case where a RealSense user with multiple L515 also tried enabling global time but found that the timestamps were not correct. In the end they used the backend timestamp

Different types of timestamp are described in the SDK documentation link below.

https://intelrealsense.github.io/librealsense/doxygen/classrs2_1_1frame.html#a25f71d45193f2f4d77960320276b83f1

The documentation describes the backend timestamp as being based upon the "host (EPOCH) clock in Kernel space".

[jiejie567]

I don't know why the backend timetamp in my window always show zero.

[MartyG-RealSense]

In regard to a zero value for the backend timestamp, as mentioned in https://github.com/IntelRealSense/librealsense/issues/7972 there have been past cases where this has been reported on devices with an Arm processor (Raspberry Pi, Nvidia Jetson, Android, etc). Are you using a computer / computing device with an Arm processor, please?

[jiejie567]

Thank you. I have succeed in enabling the backend timestamps. However, the backend timestamps seem not same.

[MartyG-RealSense]

The two L515 are at slightly different stages of their streaming according to the image. The left one is on frame 712 and the right one is on frame 822. At a speed of around 30 frames per seocnd (FPS), that would suggest that the right camera (822 frames) started streaming about 3 seconds before the left one (712 frames).

It is not possible to start two cameras at the same time in the Viewer as you have to enable one camera first and then the other one. That is probably what has happened here to cause the difference.

[MartyG-RealSense]

Hi @jiejie567 Was the information in the comment above helpful to you, please? Thanks!

[jiejie567]

I have an extra question. If I use the global time, is the time difference between their global time in line with the actual situation?

[MartyG-RealSense]

Global time corrects time-drifting between the camera and the computer by using the computer's "system time" clock. So in that sense it is changing the timing from what it would otherwise be if global time was not being used.

This correction will make the most difference when used in a librealsense application. But as ROS provides its own correction mechanism based on ROS time - as described at https://github.com/IntelRealSense/realsense-ros/issues/796#issuecomment-497997643 - the difference in RealSense ROS between having global time enabled and having it disabled may be minimal, making use of global time unnecessary. On the L515 model, global time will be disabled by default.

[MartyG-RealSense]

Hi @jiejie567 Do you require further assistance with this case, please? Thanks!

[jiejie567]

Thank you MartyG

[caobingquan6]

Hi, jiejie567. I have the same need as you, that is, I also want to be able to sync multiple L515 cameras. Have you solved the problem?

[jiejie567]

Hi, jiejie567. I have the same need as you, that is, I also want to be able to sync multiple L515 cameras. Have you solved the problem?

sorry， I have not solved it at present.