Using QR Codes to Sync Location

[dratonias]

Hello, I got an OpenXR Remoting app (using VTK) and I want to synchronize location data with an Optitrack System. So I can render things at the positions of markers in the room. My idea idea to bridge the 2 Coordinate Systems was to use QR Codes and have markers attached to them so i can calculate between the 2 Coordinate Systems. But I don't understand how to get the location data of QR Code relative to the origin of the Remoting Coordinate System (the origin is where the app connects if I observed this correctly?) using the Microsoft.MixedReality.QR Nuget package. Since the OpenXr example doesn't provide an example with QR codes and the remoting example just uses the resulting winrt::Windows::Perception::Spatial::SpatialCoordinateSystem to render there.

So the questions: How do i get the coordinates from winrt::Windows::Perception::Spatial::SpatialCoordinateSystem relative to the Origin remote Coordinate System? Is there a different (easier) way to achieve what I want to do?

If this is the wrong place to ask this any pointers to the right place would be appreciated.

[chairobl]

Hi @dratonias , I'll be looking into it and will get back to you with an answer :)

[chairobl]

Hello again @dratonias, you can use the Spatial Graph Extension to obtain a coordinate XR Space for a QR code's spatial graph node ID. I would recommend creating an anchor to get a fixed frame of reference on the HoloLens, as other XR reference spaces (e.g. local or stage spaces) may drift over time. You can find its documentation here.

[dratonias]

Hey, thank you so much for you answer. I think i mostly figured it out.

These would be the steps:

1. create the xr spatial node space struct like this: XrQuaternionf quarternion{ 0,0,0,0 }; XrVector3f vec{ 0,0,0 }; XrPosef pose{ quarternion, vec }; XrSpatialGraphNodeSpaceCreateInfoMSFT node { XR_TYPE_SPATIAL_GRAPH_NODE_SPACE_CREATE_INFO_MSFT, NULL, XR_SPATIAL_GRAPH_NODE_TYPE_STATIC_MSFT, what goes here?(I assume some acquired data from the MixedReality.QR package), pose };
2. call xrCreateSpatialGraphNodeSpaceMSFT( session, node, nodeSpace ); using my space and the current session
3. call xrLocateSpace( nodeSpace, baseSpace, currTime, location );
4. get the coordinates from location

Like mentioned in the steps I don't understand what needs to be passed in XrSpatialGraphNodeSpaceCreateInfoMSFT struct at the following place: uint8_t nodeId[XR_GUID_SIZE_MSFT]

[chairobl]

Hi @dratonias, happy to help :)

Make sure to initiate your XrQuaternionf instances to {0,0,0,1}, to properly represent no rotation (identity quaternion), as rotation quaternions are required to have length 1.

Before proceeding with your outlined steps you should first set up the QRCodeWatcher to get notified with a callback about recognized QR codes. After a QR code is recognized, you can make use of the callback's passed QRCode instance, let's call it exampleQrCode for now, to get the info you need for the XrSpatialGraphNodeSpaceCreateInfoMSFT struct.

The two fields you are missing are nodeId and pose.

The first is a global unique identifier (a.k.a. GUID or 16 byte array), representing the spatial node that is being tracked. The spatial node is your Qr Code. You can use exampleQrCode.SpatialGraphNodeId() to retrieve its node id.

For the pose you want an XrPosef defining the position and orientation of the new space’s origin within the natural reference frame of the spatial graph node. A good origin point for the QR Code would be its center, so let's create a XrPosef that represents it like this:

XrPosef exampleQrCodeSize = exampleQrCode.PhysicalSideLength();
XrPosef exampleQrCodeCenter = XrPosef{exampleQrCodeSize * 0.5,exampleQrCodeSize * 0.5,0,1};

You may then execute your steps as planned 👍

[dratonias]

Hey, just passing SpatialGraphNodeId does not seem to work. So i casted it this way:

GUID guid = args.Code().SpatialGraphNodeId();
uint8_t guidArray[16];
guidArray[0] = guid.Data1 >> 24;
guidArray[1] = guid.Data1 >> 16;
guidArray[2] = guid.Data1 >> 8;
guidArray[3] = guid.Data1;
guidArray[4] = guid.Data2 >> 8;
guidArray[5] = guid.Data2;
guidArray[6] = guid.Data3 >> 8;
guidArray[7] = guid.Data3;
for (int i = 0; i < 8; i++) 
    guidArray[8+i] = guid.Data4[i];

and passed it like this:

XrSpatialGraphNodeSpaceCreateInfoMSFT node = {
    XR_TYPE_SPATIAL_GRAPH_NODE_SPACE_CREATE_INFO_MSFT,
    nullptr,
    XR_SPATIAL_GRAPH_NODE_TYPE_STATIC_MSFT,
    guidArray[0],guidArray[1],guidArray[2],guidArray[3],guidArray[4],guidArray[5],guidArray[6],guidArray[7],
    guidArray[8],guidArray[9],guidArray[10],guidArray[11],guidArray[12],guidArray[13],guidArray[14],guidArray[15],
    pose
};

Otherwise it does not let me Compile. Is that the intended way or am I misunderstanding something?

Also when trying to get the function xrCreateSpatialGraphNodeSpaceMSFT with xrGetInstanceProcAddr, like described here https://stackoverflow.com/questions/69002900/openxr-unresolved-external-symbol, I get a -7 (XR_ERROR_FUNCTION_UNSUPPORTED). I'm using a HoloLens2 for remoting which i assume should support the function.

The code i used for this:

PFN_xrCreateSpatialGraphNodeSpaceMSFT xrCreateSpatialGraphNodeSpaceMSFT;
XrResult funciton = xrGetInstanceProcAddr(xrManager.GetXrRuntimeInstance(), "xrCreateSpatialGraphNodeSpaceMSFT", (PFN_xrVoidFunction*)&xrCreateSpatialGraphNodeSpaceMSFT);
std::cout << funciton << std::endl;

[chairobl]

Hi @dratonias , I can totally see why you'd get stuck there, apologies for skipping over that part!

We make use of memcpy(&node.nodeId, &guid, sizeof(guid)) to fill in the information :)

As for XR_ERROR_FUNCTION_UNSUPPORTED: May I ask if you have enabled the XR_MSFT_spatial_graph_bridge?

[dratonias]

I thought I had it enabled but i did not. I got it all working now. Thank you again for taking the time to help me out.

[chairobl]

Always glad to help and great to hear you got it working!