Help to Confirm Hardware Synchronization Requirements

This question is similar to #21, #23 and possibly others. I want to make Fast-LIVO work well with my custom data but I'm not sure what the synchronization requirements are - the word "synchronization" can be used differently so I hope to make it 100% clear. 🤞🤞🤞

To ask my question clearly I will use the terms "event-synchronized" and "time-synchronized" as follows - hopefully this makes sense:

Event-synchronized = 2+ events happen at the exact same moment

E.g. 2 cameras are triggered to capture 2 images simultanously but they don't necessarily share the same clock so the 2 images won't have the same timestamp even though they were taken at the same moment

Time-synchronized = 2+ sensors share the same clock / time-domain

E.g. 2 cameras have a PTP or PPS connection so their internal timestamps can be kept in-sync with eachother but they don't necessarily capture images at the exact same moment

Sometimes we need both good event-sync & time-sync but often only time-sync is required. So basically this is my question - what kind of sync do we need for Fast-LIVO to work well?

Your Datasets:

In your own data (e.g. hku2.bag) the images & lidar messages clearly have the same timestamp and you are triggering them simultaneously using the microcontroller. So in my words this data is both precisely "event-synchronized" and "time-synchronized": hku_annotated

NTU_VIRAL Datasets:

However in the NTU_VIRAL data (e.g. eee_03.bag) the images & lidar messages occur at different times but are precisly referenced to a common clock of some kind. So in my words this data is precisely "time-syncronized" but not "event-syncronized": ntu_annotated

My Datasets:

My data is more like NTU_VIRAL as my images are 15Hz and the Lidar (Mid-70) is 10Hz so they don't natually align. My data is not working very well in Fast-LIVO at the moment - obviously this could be for many reasons - but I want to make 100% sure about the syncronization first otherwise other improvements are pointless. custom_annotated

The only real difference I noticed between the config files for NTU data and your own is that the ncc_en & ncc_thre paramaters are set for NTU but not your own data - is this related? If so can you help explain a little what to do?

Hello, we have updated our synchronization scheme. Compared with the synchronization method in the LIVO paper, this new approach also sends a 1 Hz fake GPRMC message generated by the MCU to the Livox Avia LiDAR through the serial port. This ensures that the LiDAR scan and the camera frame are triggered at the same moment. Subsequently, the LiDAR scan header timestamp is assigned to the camera messages through shared memory (in the camera driver). This is why the camera and LiDAR frame header timestamps in our open-sourced data on GitHub are exactly the same; each frame of data from the camera and LiDAR is collected at the same moment (not considering exposure time and circuit delay). In addition, we have resolved the issue of timestamp rollbacks for the LiDAR, IMU, and LiDAR points, and compensated for this time in the Livox ROS driver.

hku-mars / FAST-LIVO