realtimeradio / souk-firmware

Simons Observatory UK RFSoC Firmware
GNU General Public License v3.0
3 stars 1 forks source link

timestamping #61

Open jack-h opened 1 month ago

jack-h commented 1 month ago

How does timestamp get returned with accumulators?

sr-cdf commented 3 weeks ago

Thanks, so just to make sure i'm getting this right...

There is an internal telescope time counter called "int_tt" which increments by 1 on every fpga fabric clock. (It's 64bit so it will not wrap in 2000 years with a 256MHz clock)

Telescope time registers ("tt_sync", "ext_sync_tt", "acc_tt") latch onto this counter whenever the respective sync, ext_sync, or acc_ready signals are received.

r.sync.get_tt_of_sync() gives that last stored value of "tt_sync", which was stored when ever the last sync occurred (could be a software sync).

r.sync.get_tt_of_ext_sync() gives that last stored value of "ext_sync_tt", which was stored when ever the last ext_sync occurred (could also be a software sync).

and r.accumulators[0].read_tt() gives the last stored value of "att_tt", which was stored when the last accumulation completed.

And you can directly set the "int_tt" (in fpga fabric clock units) with r.sync.load_internal_time. Then whenever the next sync arrives, this time is loaded and continues incrementing.

So i can set the tt time to the current linux time with this:

fpga_clk_hz = int(r.rfdc.core.get_fabric_clk_freq(0,0)*1e6)
print(fpga_clk_hz)
ttnow = int( time.time() * fpga_clk_hz) 
r.sync.load_internal_time( ttnow )
r.sync.arm_sync(wait=False)
r.sync.sw_sync()

Then whenever I call accumulator.read_tt(), I get the tt_time, which should pretty much equal the linux time.

Except the linux time and fpga time seem to be on different clocks, and are slowly drifting apart? And there was some delay between reading the Linux time and loading/triggering the internal time.

But if we have a PPS signal, we can get a better time with something like this:


# every once in while make sure to update the tt_time:
fpga_clk_hz = r.rfdc.core.get_fabric_clk_freq(0,0)*1e6 
r.sync.load_internal_time( int(time.time() * fpga_clk_hz) )
r.sync.arm_sync(wait=False)
r.sync.sw_sync()

#then assuming a PPS every second on the sync input
clocks_since_pps = r.accumulators[0].read_tt() - r.sync.get_tt_of_sync()
time_since_pps = clocks_since_pps / fpga_clk_hz
time_of_pps = r.sync.get_tt_of_sync() /fpga_clk_hz
better_time = time_of_pps + time_since_pps

And there are probably more ways to get even better times.

Does that all sound about right?

sr-cdf commented 3 weeks ago

Looks like getting the time takes 0.2ms, load_internal_time takes 23ms, sw_sync takes 66ms

r.sync.arm_sync(wait=False)
fpga_clk_hz = int(r.rfdc.core.get_fabric_clk_freq(0,0)*1e6)
print(fpga_clk_hz)

ttnow = int( time.time() * fpga_clk_hz)
print(time.time()-ttnow/fpga_clk_hz)

r.sync.load_internal_time( ttnow )
print(time.time()-ttnow/fpga_clk_hz)

r.sync.sw_sync()
print(time.time()-ttnow/fpga_clk_hz)
print(time.time()-ttnow/fpga_clk_hz)

256000000
0.00023674964904785156
0.023051738739013672
0.08935379981994629
0.08971047401428223
jack-h commented 2 weeks ago

Everything in your post is impressively correct, I believe.

With no PPS supplied, the accuracy of the internal telescope time is limited to how quickly various operations can be completed. What you're seeing -- a few tens of milliseconds -- seems plausible.

Provided the RFSoC is clocking from a GPS-locked reference (rather than the onboard XO) it shouldn't drift vs NTP.

The (my) typical load mechanism is to-be-implemented in https://github.com/realtimeradio/souk-firmware/blob/8dc9c36529073f3e2cf8b86f421a34ea664af0c8/software/control_sw/src/blocks/sync.py#L246

It amounts to:

  1. Wait for a PPS to pass
  2. Calculate the appropriate TT value at the next PPS
  3. Arm sync logic to load this time on the next PPS
  4. Check no PPS has passed yet, since step 1.
  5. Wait for PPS

Provided steps 2&3 take less than 1 second this should reliably lock TT to the PPS reference.