ros-drivers / flir_camera_driver

153 stars 145 forks source link

[Humble]Timestamp syncing with Lidar and IMU #188

Open li-dahua opened 1 week ago

li-dahua commented 1 week ago

Hi,I would want to sync the time stamp of a stereo of blackfly s cameras with IMU and Lidar. But I found the time stamp is output by the camera sdk directly. How can I set timestamp? Thank you.

It seems these parameter do not allow me to change it. 'adjust_timestamp': True, 'chunk_mode_active': True, 'chunk_selector_timestamp': 'Timestamp', 'chunk_enable_timestamp': True,

System details

berndpfrommer commented 1 week ago

The camera driver has no way to get a time stamp from the IMU or the lidar. But what you can do is trigger the cameras with a hardware pulse signal from the IMU (if you have an IMU that has a sync out pulse). So this way you know that the camera frames correspond to one of the IMU frames, but you still don't know which of the IMU frames triggered the camera frames, and there is no easy way to figure this out. Certain calibration softwares (for instance Kalibr) can figure out what the delay is by comparing camera and IMU motion, that's the only way I'm aware of. Alternatively, you can set the IMU to really low trigger frequencies such that you can manually see what the delay is between IMU and camera frames. Either way you do it, you still have to hope and pray that the delay remains constant across data sets. I have not ever synchronized a LIDAR to other sensors, so I cannot help you with that task. If you run the synchronized_camera_driver it will figure out which of the stereo camera frames belong together, and put the same header time stamps on them. To do that it has to guess which frames were generated by the same sync pulse. Between two cameras that is relatively easy because the frames are usually well separated in time. With an IMU at high frequency this is harder.

li-dahua commented 1 week ago

Is it possible to sync Blackfly S GigE and other devices using PTP? I have 2 more GigE cameras and a livox Lidar with built-in IMU that allows PTP synchronization. https://github.com/Livox-SDK/Livox-SDK/wiki/livox-device-time-synchronization-manual

berndpfrommer commented 1 week ago

Ok, the blackflys gige can do it. I don't have that camera though, only USB3

berndpfrommer commented 1 week ago

I can't find any documentation about how PTP is supposed to be used with the Spinnaker SDK. I suspect ptp will simply affect chunk time. I suggest you use Spinview to enable PTP somehow. Then trigger the cameras externally from the lidar or another pulse generator. Enable chunk mode and chunk time stamps. Once you have all of that working, hack the ros driver to print out the raw chunk time stamp, and compare between the two cameras. Are they almost identical? Related to epoch time? If yes then we can modify the driver to use the chunk time directly.