Time synchronization issue

[hyonlim]

Dear Team Ouster, We've followed Software User Guide v1.12.0, Section 9. PTP Quickstart Guide, but never succeeded to get proper time sync on ROS.

https://github.com/ouster-lidar/ouster_example/issues/144 https://github.com/ouster-lidar/ouster_example/issues/78

There were actually several cases ended with no definitive answer on this question.

Problem is, even though we follow the guide and confirms ptp is working by given method Section 9.5, ROS always shows 36 (or 37) seconds delay.

$ rostopic delay /topic_name

It seems leap second problem, but "-O 0" option was not working.

Team Ouster should test this and report result here, otherwise, this will be significant customer issue.

[Krishtof-Korda]

Hi @hyonlim,

This seems to be an issue with Linux PTP not being configured correctly. The sensor will only use the “current_utc_offset”: 37 if ptp4l is transmitting "current_utc_offset_valid": 1 and the offset is configured correctly.

If ptp4l transmits "current_utc_offset_valid": 0 the sensor will ignore the offset since it is reported as not valid.

Can you please try the solution mentioned in this forum post? Thanks!

[mauricefallon]

Hi @hyonlim and @Krishtof-Korda , Is there as possibility that v2.0 has added a regression which broke the PTP server? We see the same offset issue - timesync is drifting out to 36 seconds (measuring with rostopic delay).

This is with v2.0 firmware on an OS0-64 and the driver here on github.

We had been using v1.13 of the firmware and the beta ROS driver previously without this problem.

[Krishtof-Korda]

Hi @mauricefallon, There are no regressions that we know of. The PTP timing portion of the firmware only had corrections for leap year. Please check that ptpt4l is transmitting the required fields, as mentioned above.

[Krishtof-Korda]

Hi all,

For those having this issue, can you please show the results from the command below?

pmc -u -b 0 'get TIME_PROPERTIES_DATA_SET'

Example response:

sending: GET TIME_PROPERTIES_DATA_SET
0cc47a.fffe.3a2c5c-0 seq 0 RESPONSE MANAGMENT TIME_PROPERTIES_DATA_SET
currentUtcOffset 36
leap61 0
leap59 0
currentUtcOffsetValid 1
ptpTimescale 1
timeTraceable 0
frequencyTraceable 0
timeSource 0xa0

You can read more on this forum posting on how to fix the issue with your GM not sending the correct currentUtcOffset or currentUtcOffsetValid 1. If currentUtcOffsetValid is 0 then the Ouster sensor will ignore the currentUtcOffset since it is not valid.

[Hommus]

@Krishtof-Korda I've followed the discussions in #78 and #144 and have ended up in the same place as @hyonlim. From what I can tell, chrony, ptp4l and phc2sys are setup correctly, but there are 2 issues:

1. The ouster lidars have a 37s average delay (recorded by ros2 topic delay), but other non-ouster sensors in my network are not affected (delays in the milliseconds).
2. pmc returns currentUtcOffsetValid = 0 which I assume is related to the ouster lidar issue?

$ sudo phc_ctl eth0 cmp
phc_ctl[445946.126]: offset from CLOCK_REALTIME is 5649ns

$ sudo pmc -u -b 0 'get TIME_PROPERTIES_DATA_SET'
sending: GET TIME_PROPERTIES_DATA_SET
    48b02d.fffe.dcb463-0 seq 0 RESPONSE MANAGEMENT TIME_PROPERTIES_DATA_SET 
        currentUtcOffset      37
        leap61                0
        leap59                0
        currentUtcOffsetValid 0
        ptpTimescale          1
        timeTraceable         0
        frequencyTraceable    0
        timeSource            0xa0

When implementing the solutions from the comments above:

• all other sensors now have a delay of -37s
• the lidar topics in ros2 start returning a delays of 30s and 25s respectively
• phc breaks:

$ sudo phc_ctl eth0 cmp
phc_ctl[447081.120]: offset from CLOCK_REALTIME is -36999999547ns

[Hommus]

Following up my comment from yesterday, I found a fix in https://github.com/ouster-lidar/ouster-ros/issues/231: You have to update the ptp_utc_tai_offset parameter to 0.0.