search

thaytan / OpenHMD

Free and Open Source API and drivers for immersive technology.
Boost Software License 1.0
142 stars 11 forks source link

Correlate Rift HMD time with frame captures #4

Closed thaytan closed 3 years ago

thaytan commented 4 years ago

In order to properly combine data from IMU observations with camera tracking, we need a common set of timestamps.

The code that associates the led-pattern-phase with an incoming camera frame could also pass the IMU timestamp to associate that with the video frame.

It will need testing whether that 1ms granularity of 1000Hz IMU materially affects the results. If so, some smoothing and interpolation using the system clock could help.

thaytan commented 3 years ago

The way I'm handling timestamps has changed a bit since I filed this issue. In the filtering code, the internal device times are used directly to avoid jitter. There are still a few places where it might be useful to have a tracker correlating and interpolating device times to local system time - at least for detecting gaps / skips in the data.

thaytan commented 3 years ago

As long as IMU data is processed in a timely fashion, this is all working now. Exposure notifications are used to store the expected state of the world, and they are correlated well with USB camera data, so that each frame has a good prior for where the system expects each device to be in that frame.