ROS timestamps of frames would match non-matching (consecutive) frames

[bmegli]

Problem

With standalone sync mode and sync enabled, for Femto Bolt:

• printing both system host receive timestamp (used for ROS) and hardware timestamp

I can see

• matching pairs of frames by hardware timestamp published (good)

But ROS timestamps (based on system timestamp)

• from different frame pairs are much closer to each other than the corresponding frames (bad)

Log

• each pair of frames is synced (1-2 ms difference) by hardware timestamp (good)
• each pair of frames diffs by system (and ROS) timestamp by around 40-60 ms (bad)
• matching in system time domain (and ROS) one would get wrong pairs
  • e.g. second RGB is closer closer to first depth than to second depth

[ INFO] [1706867703.006876149]: publishing hw: 1706867702863 sys 1706867702955
[ INFO] [1706867703.012360773]: publishing hw: 1706867702862 sys 1706867703006
[ INFO] [1706867703.074237997]: publishing hw: 1706867702930 sys 1706867703021
[ INFO] [1706867703.079670789]: publishing hw: 1706867702928 sys 1706867703074
[ INFO] [1706867703.130402564]: publishing hw: 1706867702996 sys 1706867703090
[ INFO] [1706867703.136386807]: publishing hw: 1706867702995 sys 1706867703130
[ INFO] [1706867703.196850258]: publishing hw: 1706867703063 sys 1706867703153
[ INFO] [1706867703.202784873]: publishing hw: 1706867703061 sys 1706867703196
[ INFO] [1706867703.268265276]: publishing hw: 1706867703129 sys 1706867703220
[ INFO] [1706867703.273795934]: publishing hw: 1706867703128 sys 1706867703268
[ INFO] [1706867703.330562980]: publishing hw: 1706867703196 sys 1706867703289

Code to print timestamps

void OBCameraNode::onNewFrameCallback(const std::shared_ptr<ob::Frame>& frame,
   int height = static_cast<int>(video_frame->height());

   auto timestamp = frameTimeStampToROSTime(video_frame->systemTimeStamp());
+  auto hw_timestamp = video_frame->timeStamp();
+  auto sys_timestamp = video_frame->systemTimeStamp();
+  auto diff = hw_timestamp - sys_timestamp;
+
+  ROS_INFO_STREAM("publishing" <<  " hw: " << hw_timestamp << " sys " << sys_timestamp);
+
   if (!camera_params_) {
     camera_params_ = pipeline_->getCameraParam();
   }

Launchfile params to enable sync

+    <arg name="enable_frame_sync" value="true"/>
+    <arg name="sync_mode"         value="STANDALONE"/>    

[bmegli]

I can see still hot commit:

• https://github.com/orbbec/OrbbecSDK_ROS1/commit/088bf2285771fca86a1da83659d56d2da95bbed9

That can partially fix the problem by switching to hardware timestamp. Hopefully it doesn't create other.

[bmegli]

The underlying problem probably remains, in the sense that host system receives pair of frames with considerable delay (above experiment was run at 15 fps)