Understand PTP and its mode
packet timestamping and how the kernel supports it
Why HW timestamp
Device driver support of PTP
Background
Event ordering is essential to some applications: Transactions, Logs, Debugging and performance analysis
Ordering is based on TS
collected from large ranges of machines
Need for clocks sync on local network: frequency, phase, time
NTP
Network time protocol is a protocol for clock sync
Provides accuracy within a few ms
Not precise
Need higher PTP
Overview PTP
sub ms accuracy on local network
IEEE 1588-2002 2008 2019
PTP hierarchy 1/3
Hierachical leader
Leader "grandmaster"
boundary, follower("slave") clocks
PTP packets transmitted over Eth or UDP
Leader clock:
Time src for PTP
sync it with external src GNSS
ordinary clock
Bounday clock:
optional
multiple PTP newtowk connections and relay accurate time
sync its clk against leader
acts as clock src for the followers
May become the leader if the current leader disappear
Follower clock:
Synchronize its clock to a leader
May become the leader if the leader disappear
Is an "ordinary clock"
PTP sync mechanism
Time offset is computed based on TS of packet sent and received.
Done by the follower.
Timestamps made on the leader side are sent to follower by follow up packets(follow_up, delay_resp)
The trip time include the transmit time and the delta between the two clocks
d if = t1-t0 + deltat
d fi = t3-t2 + deltat
We assume the two trip times are equal, hence we have:
deltat = 1/2(t1-t0+t2-t3)
One step vs two step
PTP can work in two operating modes: 1-step and 2-step
1-step includes t0 in the sync packet.
There is no follow_up packet
The diff lies in the leader side. It needs a hardware enabled device to include t0 in the sync packets
All followers should support both modes: there is no hardware requirement for receiving 1-step sync packets
Packet timestamping
Timestamp accuracy
Packet timestamps, when used for PTP, must be accurate: they play a critical role in the time offset computation
Ideally we would like a timestamp issued at the exact time of transmission, when the packet leave the device
Not possible in the real world, the timestamp has to occur before
Two possibilities: in software and in hardware
Software timestamping
Timestamp is done in the app or in the kernel
Uses the system clock
Error and deltas are big:
Timestamp is done far away from the actual transmission
A lot can interface: scheduling, queuing, interrupts...
Hardware timestamping
Timestamp is done in the hardware
done in the MAC or in the PHY or using a dedicated controller
Uses a PTP hardware clock (PHC)
Error and deltas are small
Timestamp occurs close to the actual transmission
The packet is already in the hardware
PTP offloading support in Linux
Overview
Two mechanisms are combined to provide support for offloading PTP packets timestamping:
1 SO_TIMESTAMPING socket option
2 The PTP hardware clock PHC infrastructure
Read the full doc at: Documentation/networking/timestamping.rst
SO_TIMESTAMPING
Config using setsockopt
Generates timestamps on reception, transmission or both
Works for streams and datagrams
Support multiple timestamp src
SOF_TS_RX_SOFTWARE
SOF_TS_TX_SOFTWARE
SOF_TS_RX_HARDWARE
SOF_TS_TX_HARDWARE
and two other options not used for PTP app
SOF... TX_SCHED
SOF_TS_TX_ACK
Hardware timestamping must be inited for each device driver expected used
Configuration passed using the SIOCSHWTSTAMP ioctl, Must choose a tx_type and an rx_filter
Possible values of tx_type
Possible values for rx_filter
see include/uapi/linux/net_tstamp.h
ethtool -T eth0
Supporting SO_TS in a device driver
In a networking eth driver MAC implementing
get_ts_info in struct ethtool_ops
ndo_do_ioctl in struct net_device_ops for SIOCSHWTSTAMP and SIOCGHWTSTAMP
Filling hwtstamps in struct skbuff with Rx timestamps
Calling skb_tstamp_tx() when a Tx timestamp is reported by the hardware
In networking PHY or dedicted engines driver: impl
struct mii_timestamper callbacks in struct phy_device
ts_info
hwtstamp
rxtstamp
txtstamp and calling skb_complete_tx_timestamp()
Both interfaces allow us to:
1 report ts capabilities: ts_info and get_ts_info
2 Config the mode to use hwtstamp and ndo_do_ioctl
3 Report Rx timestamps (rxtstamp and hwtstamps)
4 Report Tx timestamps (txtstamp/skb_complete_tx_timestamp() and skb_tstamp_tx)
PTP hardware clock
PHC are used by hardware engines to ts packets
PHC must be synchronized
Described by struct ptp_clock_info which embeds operation callbacks:
gettimx64
settime64
adjfine adjusts the frequency of the hardware clock
adjtime shifts the time of the hardware clock by an s64 delta
adjphase adjust the phase of the HC by an s32 phase
max_adj---- Maximum frequency adjustment in parts per billion
IEEE-1588
Richard Cochran
PHC and timestamping
ptp4l and phc2sys
ptp4l
Impl of PTP for odinary and boundary clocks
can use SW or HW timestamping
can perform PTP operation on top of UDP
can optionally use a config file
ptp4l -i eth0 -H -2 -m
phc2sys
Sync two or more clocks
keep the system clock in sync with the PHC
when using hardware timestamping, ptp4l adjust the PHC and then phc2sys adjust the system clock
When using software timestamping, phc2sys isn't used, system clk is directly adjusted by ptp4l
raychenv
commented
2 years ago
https://thebroadcastknowledge.com/tag/timing/ Antoine Tenart Bootlin
Understand PTP and its mode packet timestamping and how the kernel supports it Why HW timestamp Device driver support of PTP
Background
Event ordering is essential to some applications: Transactions, Logs, Debugging and performance analysis Ordering is based on TS collected from large ranges of machines Need for clocks sync on local network: frequency, phase, time
NTP Network time protocol is a protocol for clock sync Provides accuracy within a few ms Not precise Need higher PTP
Overview PTP
sub ms accuracy on local network IEEE 1588-2002 2008 2019
Leader clock: Time src for PTP sync it with external src GNSS ordinary clock
Bounday clock: optional multiple PTP newtowk connections and relay accurate time sync its clk against leader acts as clock src for the followers May become the leader if the current leader disappear
Follower clock: Synchronize its clock to a leader May become the leader if the leader disappear Is an "ordinary clock"
Time offset is computed based on TS of packet sent and received. Done by the follower. Timestamps made on the leader side are sent to follower by follow up packets(follow_up, delay_resp)
The trip time include the transmit time and the delta between the two clocks d if = t1-t0 + deltat d fi = t3-t2 + deltat We assume the two trip times are equal, hence we have: deltat = 1/2(t1-t0+t2-t3)
One step vs two step PTP can work in two operating modes: 1-step and 2-step 1-step includes t0 in the sync packet. There is no follow_up packet The diff lies in the leader side. It needs a hardware enabled device to include t0 in the sync packets All followers should support both modes: there is no hardware requirement for receiving 1-step sync packets
Packet timestamping
Timestamp accuracy Packet timestamps, when used for PTP, must be accurate: they play a critical role in the time offset computation Ideally we would like a timestamp issued at the exact time of transmission, when the packet leave the device Not possible in the real world, the timestamp has to occur before Two possibilities: in software and in hardware
Software timestamping Timestamp is done in the app or in the kernel Uses the system clock Error and deltas are big: Timestamp is done far away from the actual transmission A lot can interface: scheduling, queuing, interrupts...
Hardware timestamping Timestamp is done in the hardware done in the MAC or in the PHY or using a dedicated controller Uses a PTP hardware clock (PHC) Error and deltas are small Timestamp occurs close to the actual transmission The packet is already in the hardware
PTP offloading support in Linux
Read the full doc at: Documentation/networking/timestamping.rst
Hardware timestamping must be inited for each device driver expected used Configuration passed using the SIOCSHWTSTAMP ioctl, Must choose a tx_type and an rx_filter Possible values of tx_type Possible values for rx_filter see include/uapi/linux/net_tstamp.h
ethtool -T eth0
In networking PHY or dedicted engines driver: impl struct mii_timestamper callbacks in struct phy_device ts_info hwtstamp rxtstamp txtstamp and calling skb_complete_tx_timestamp()
Both interfaces allow us to: 1 report ts capabilities: ts_info and get_ts_info 2 Config the mode to use hwtstamp and ndo_do_ioctl 3 Report Rx timestamps (rxtstamp and hwtstamps) 4 Report Tx timestamps (txtstamp/skb_complete_tx_timestamp() and skb_tstamp_tx)
max_adj---- Maximum frequency adjustment in parts per billion
network switch: drivers/net/ethernet/mscc/ PHY drivers/net/phy/mscc/mscc_ptp.c
User-space PTP impl
Linux PTP
IEEE-1588 Richard Cochran PHC and timestamping ptp4l and phc2sys
ptp4l
Impl of PTP for odinary and boundary clocks can use SW or HW timestamping can perform PTP operation on top of UDP can optionally use a config file ptp4l -i eth0 -H -2 -m
phc2sys
Sync two or more clocks keep the system clock in sync with the PHC when using hardware timestamping, ptp4l adjust the PHC and then phc2sys adjust the system clock When using software timestamping, phc2sys isn't used, system clk is directly adjusted by ptp4l