[REQ] Incorporate clock device<->host offsets

[tstenner]

There's been a review of LSL discussion here that pointed out one problem with LSL: the latency between the inlet and outlet is well accounted for, but there's also some varying latency between the actual device and the host.

For directly connected devices, this latency is typically quite small and has almost no jitter. Wireless connections to the devices (i.e. Bluetooth and Wifi), on the other hand, can add a significant amount of jitter. In an ideal world, LSL would run directly on the device, but the requirement to store up to 6 minutes of data per client in case it reconnects is quite steep and (other than some ESP32 devices?) the C++/ASIO support is limited at best.

One way to work around this would be a separate channel that transmits the round-trip times from the host to the device and (if present) offsets (the TobiiPro devices calculate their own clock offsets that – on Windows and Linux at least – use the same OS clock as LSL, but others might reference an entirely different hardware clock), so that some variance in timestamps can be accounted for and if the device has its own timestamps these can be used with LSL.

[agricolab]

While i believe that i understand the problem, I do not fully understand the solution. You wrote "One way to work around this would be a separate channel that transmits the round-trip times [and] offsets". But a separate channel would be configured in the outlet code, right? I guess i misunderstand sth. here. I also guess we agree that there is the other challenge that not all devices support round-trips, or that what constitutes a round-trip is different for different devices.

So wouldn't a solution for this better live in the respective apps?

[tstenner]

But a separate channel would be configured in the outlet code, right?

Exactly. There's already a mechanism to transmit offsets (namely, outlet<->inlet), so something like a periodic t0 = pylsl.local_clock(); device_clock=query_device_clock(); t1=pylsl.local_clock(); outlet.transmit_device_offset((t1-t0/2), device_clock, t1-t0); could be used after recording the data.

I also guess we agree that there is the other challenge that not all devices support round-trips, or that what constitutes a round-trip is different for different devices.

Some devices give both offsets and accuracy (e.g. the TobiiPro devices), others only the RTT (e.g. by timing a call to a simple API function). Of course, the RTT itself isn't that helpful unless it's stable and the transmission times are equal in both directions, but for devices with an open firmware it could be implemented relatively easy.

So wouldn't a solution for this better live in the respective apps?

This has to be done in the apps, but without support in liblsl it's next to impossible to get everyone to do it uniformly.

[agricolab]

I think i understand, and agree that LSL support for measuring device offset/latency/jitter is beneficial.

One architecture that jumps into my mind is a wrapper function, which takes a function that allows such an estimate and returns the offset/latency/jitter based on local_clock.

Now if there is open firmware, but it does not support a full LSL outlet: One option could be that liblsl supports a lite outlet, maybe let's call it an outpost, which receives calls and returns them to allow to measure offset/jitter/latency along the NTP.