I want to locate with xrLocateSpace HoloLens 2 PV camera at given frame timestamps. But I don't know how to properly interop between camera timestamps and OpenXR runtime timestamps, such as predicted display times.
Below are the logs of a running session where I'm outputting the timestamps of interest:
MediaFrameReference time is the timestamp returned as-is by MediaFrameReference::SystemRelativeTime in MediaFrameReader::FrameArrived event;
System QPC time is the timestamp returned by a call to QueryPerformanceCounter in MediaFrameReader::FrameArrived event;
System XrTime is the above timestamp converted to XrTime with xrConvertWin32PerformanceCounterToTimeKHR;
OpenXR predictedDisplayTime is OpenXR runtime XrTime timestamp.
MediaFrameReference time = 541897815709
System QPC time = 1040444504957
System XrTime = 54189817966510
OpenXR predictedDisplayTime = 54189878473400
OpenXR predictedDisplayTime = 54189895134200
MediaFrameReference time = 541898148908
System QPC time = 1040445144667
System XrTime = 54189851284739
OpenXR predictedDisplayTime = 54189911812200
OpenXR predictedDisplayTime = 54189928482900
MediaFrameReference time = 541898482108
System QPC time = 1040445778477
System XrTime = 54189884295677
OpenXR predictedDisplayTime = 54189945160400
OpenXR predictedDisplayTime = 54189961798100
MediaFrameReference time = 541898815307
System QPC time = 1040446429967
System XrTime = 54189918227447
OpenXR predictedDisplayTime = 54189978474100
OpenXR predictedDisplayTime = 54189995152100
MediaFrameReference time = 541899148507
System QPC time = 1040447063824
System XrTime = 54189951240833
OpenXR predictedDisplayTime = 54190011809100
OpenXR predictedDisplayTime = 54190028465800
If I pass to xrLocateSpace the timestamp returned by MediaFrameReference::SystemRelativeTime, the PV camera isn't located. Expected as it's a QPC time value.
Not shown in the logs, if I pass to xrLocateSpace the above timestamp converted to XrTime with xrConvertWin32PerformanceCounterToTimeKHR, the PV camera isn't located. The resulting XrTime timestamp appears to be based on a very different time reference than OpenXR predicted display time.
If I pass to xrLocateSpace the timestamp returned by the call to QueryPerformanceCounter in MediaFrameReader::FrameArrived event, the PV camera isn't located. Expected as it's a QPC time value.
But if I pass to xrLocateSpace the above timestamp converted to XrTime with xrConvertWin32PerformanceCounterToTimeKHR, the PV camera is successfully located.
I wish I could use the camera timestamp directly (using the appropriate adjustment and/or conversion, of course) rather than having to perform a call to QueryPerformanceCounter to generate a timestamp suitable for use with xrLocateSpace (once converted to XrTime). Any guidance on this or is calling QueryPerformanceCounter in MediaFrameReader::FrameArrived event the right approach? Otherwise, what am I doing wrong with the camera timestamps handling?
If you carefully look at the timestamps returned by MediaFrameReference::SystemRelativeTime, you'll notice there are roughly 1/100th of OpenXR runtime XrTime timestamps. And indeed, if I pass to xrLocateSpace the timestamps returned by MediaFrameReference::SystemRelativeTime multiplied by 100, the PV camera is successfully located. Is this 100 multiplier in any way correlated to the 100ns interval QPC time in MediaFrameReference::SystemRelativeTime documentation? In short, is it expected that I simply have to multiply by 100 the timestamps returned by MediaFrameReference::SystemRelativeTime to pass to xrLocateSpace without having to rely upon xrConvertWin32PerformanceCounterToTimeKHR at all? Does it mean that HoloLens 2 PV camera gives already-suitable-for-use-with-OpenXR timestamps? If yes, is this guaranteed to always be the case for all the WMR devices? What's then the purpose of XR_KHR_win32_convert_performance_counter_time extension?
Hi,
I want to locate with
xrLocateSpace
HoloLens 2 PV camera at given frame timestamps. But I don't know how to properly interop between camera timestamps and OpenXR runtime timestamps, such as predicted display times.For HoloLens 2 PV camera, I'm using MediaFrameReference::SystemRelativeTime timestamps, defined as 100ns interval QPC time, so a fraction of a QPC value if I understand it correctly. Thanks to xrConvertWin32PerformanceCounterToTimeKHR from XR_KHR_win32_convert_performance_counter_time extension, I was hoping to convert the camera timestamps to OpenXR runtime
XrTime
timestamps for use withxrLocateSpace
. But this doesn't work, i.e. I'm getting an invalid location.Below are the logs of a running session where I'm outputting the timestamps of interest:
MediaFrameReference::SystemRelativeTime
inMediaFrameReader::FrameArrived
event;QueryPerformanceCounter
inMediaFrameReader::FrameArrived
event;XrTime
withxrConvertWin32PerformanceCounterToTimeKHR
;XrTime
timestamp.If I pass to
xrLocateSpace
the timestamp returned byMediaFrameReference::SystemRelativeTime
, the PV camera isn't located. Expected as it's a QPC time value.Not shown in the logs, if I pass to
xrLocateSpace
the above timestamp converted toXrTime
withxrConvertWin32PerformanceCounterToTimeKHR
, the PV camera isn't located. The resultingXrTime
timestamp appears to be based on a very different time reference than OpenXR predicted display time.If I pass to
xrLocateSpace
the timestamp returned by the call toQueryPerformanceCounter
inMediaFrameReader::FrameArrived
event, the PV camera isn't located. Expected as it's a QPC time value.But if I pass to
xrLocateSpace
the above timestamp converted toXrTime
withxrConvertWin32PerformanceCounterToTimeKHR
, the PV camera is successfully located.I wish I could use the camera timestamp directly (using the appropriate adjustment and/or conversion, of course) rather than having to perform a call to
QueryPerformanceCounter
to generate a timestamp suitable for use withxrLocateSpace
(once converted toXrTime
). Any guidance on this or is callingQueryPerformanceCounter
inMediaFrameReader::FrameArrived
event the right approach? Otherwise, what am I doing wrong with the camera timestamps handling?If you carefully look at the timestamps returned by
MediaFrameReference::SystemRelativeTime
, you'll notice there are roughly 1/100th of OpenXR runtimeXrTime
timestamps. And indeed, if I pass toxrLocateSpace
the timestamps returned byMediaFrameReference::SystemRelativeTime
multiplied by 100, the PV camera is successfully located. Is this 100 multiplier in any way correlated to the 100ns interval QPC time inMediaFrameReference::SystemRelativeTime
documentation? In short, is it expected that I simply have to multiply by 100 the timestamps returned byMediaFrameReference::SystemRelativeTime
to pass toxrLocateSpace
without having to rely uponxrConvertWin32PerformanceCounterToTimeKHR
at all? Does it mean that HoloLens 2 PV camera gives already-suitable-for-use-with-OpenXR timestamps? If yes, is this guaranteed to always be the case for all the WMR devices? What's then the purpose of XR_KHR_win32_convert_performance_counter_time extension?Thanks.