[Feature-Request] Add timestamp syncing in syncTimerCB of Stereo Node

[michaelnguyen11]

Start with the why:

The syncTimerCB in stereo node uses sequence number syncing for the rectified left and right frames. I would like to have timestamp syncing in the stereo node when i_publish_synced_rect_pair is enabled.

Move to the what:

I'm using OAK-FFC-4P + AR0234 stereo pair. Right now, the OAK-FFC-4P have not had hardware synchronization yet, debate is still continue. So I'm using software synchronization which is enabled by default in DepthAI library. The rectified left and right frame on my OAK-FFC-4P are not matching in term of sequence number, but it's matching perfectly in term of timestamp. Mr. Erik in Luxonis forum said it's expected behavior, and suggest me to open a feature-request to depthai-ros

frameRectLeft seq: 1, ts: 5:46:22.442598
frameRectRight seq: 0, ts: 5:46:22.442610
------------------- 
frameRectLeft seq: 2, ts: 5:46:22.475933
frameRectRight seq: 1, ts: 5:46:22.475954
------------------- 
frameRectLeft seq: 3, ts: 5:46:22.509263
frameRectRight seq: 2, ts: 5:46:22.509279

Move to the how:

Add timestamp syncing option in syncTimerCB of Stereo Node. My idea is compare left and right timestamp, if it is less than sub-ms, then the Left and right rectified frames are synchronized. For example :

if(left->getSequenceNum() != right->getSequenceNum()) {
        RCLCPP_WARN(getROSNode()->get_logger(), "Left and right rectified frames are not synchronized!");
}
else if (std::abs(left->getTimestamp().total_seconds() - right->getTimestamp().total_seconds()) >= 0.001) {
        RCLCPP_WARN(getROSNode()->get_logger(), "Left and right rectified frames are not synchronized!");
}
else {
...
}

[michaelnguyen11]

I print that the time difference between rectified left and right frames is around 7–13us.

For my case, to avoid some peak in a different timestamp, I put the condition in less than 100us.

-    if(left->getSequenceNum() != right->getSequenceNum()) {
+    if(left->getSequenceNum() != right->getSequenceNum() && std::chrono::duration_cast<std::chrono::microseconds>(right->getTimestamp() - left->getTimestamp()).count() >= 100) {
         RCLCPP_WARN(getROSNode()->get_logger(), "Left and right rectified frames are not synchronized!");
     } else {

[Serafadam]

Hi, thanks for the feature idea, for now it's hard to estimate ETA on that, but feel free to open a PR with the changes, from my side following things need to be considered:

• Adding a parameter to choose whether to sync based on the Sequence number or a timestamp
• This needs to be checked but I assume that the timestamp difference might need to be tuned based on camera FPS Additionally I wanted to mention that in near future (not sure yet on the ETA) a Sync node will be added in depthai-core that will allow easier syncing of multiple messages, after that is available it will be added to depthai_ros_driver