ALVR Async Reprojection Function

[leedc20]

This is not a bug or issue report. Just studying and learning ALVR features and code base.
I have read that ALVR used Async Reprojection technology, so excited. I'm trying to find the relevant source code part that performs reprojection. I assume the reprojection is performed based on Pose information submitted by vrcompositor. So, looking at the OvrDirectModeComponent::SubmitLayer() function.

void OvrDirectModeComponent::SubmitLayer(const SubmitLayerPerEye_t(&perEye)[2])
{
    auto pPose = &perEye[0].mHmdPose; // TODO: are both poses the same? Name HMD suggests yes.
    Debug("SubmitLayer Handles=%p,%p DepthHandles=%p,%p %f-%f,%f-%f %f-%f,%f-%f\n%f,%f,%f,%f\n%f,%f,%f,%f\n%f,%f,%f,%f\n"
        , perEye[0].hTexture, perEye[1].hTexture, perEye[0].hDepthTexture, perEye[1].hDepthTexture
        , perEye[0].bounds.uMin, perEye[0].bounds.uMax, perEye[0].bounds.vMin, perEye[0].bounds.vMax
        , perEye[1].bounds.uMin, perEye[1].bounds.uMax, perEye[1].bounds.vMin, perEye[1].bounds.vMax
        , pPose->m[0][0], pPose->m[0][1], pPose->m[0][2], pPose->m[0][3]
        , pPose->m[1][0], pPose->m[1][1], pPose->m[1][2], pPose->m[1][3]
        , pPose->m[2][0], pPose->m[2][1], pPose->m[2][2], pPose->m[2][3]
    );
    // pPose is qRotation which is calculated by SteamVR using vr::DriverPose_t::qRotation.
    // pPose->m[0][0], pPose->m[0][1], pPose->m[0][2],
    // pPose->m[1][0], pPose->m[1][1], pPose->m[1][2], 
    // pPose->m[2][0], pPose->m[2][1], pPose->m[2][2], 
    // position
    // x = pPose->m[0][3], y = pPose->m[1][3], z = pPose->m[2][3]

    if (m_submitLayer == 0) {
        // Detect FrameIndex of submitted frame by pPose.
        // This is important part to achieve smooth headtracking.
        // We search for history of TrackingInfo and find the TrackingInfo which have nearest matrix value.

        auto pose = m_poseHistory->GetBestPoseMatch(*pPose);
        if (pose) {
            // found the frameIndex
            m_prevTargetTimestampNs = m_targetTimestampNs;
            m_targetTimestampNs = pose->info.targetTimestampNs;

            m_prevFramePoseRotation = m_framePoseRotation;
            m_framePoseRotation.x = pose->info.HeadPose_Pose_Orientation.x;
            m_framePoseRotation.y = pose->info.HeadPose_Pose_Orientation.y;
            m_framePoseRotation.z = pose->info.HeadPose_Pose_Orientation.z;
            m_framePoseRotation.w = pose->info.HeadPose_Pose_Orientation.w;

            Debug("Frame pose found. m_prevSubmitFrameIndex=%llu m_submitFrameIndex=%llu\n", m_prevTargetTimestampNs, m_targetTimestampNs);
        }
        else {
            m_targetTimestampNs = 0;
            m_framePoseRotation = HmdQuaternion_Init(0.0, 0.0, 0.0, 0.0);
        }
    }
    if (m_submitLayer < MAX_LAYERS) {
        m_submitLayers[m_submitLayer][0] = perEye[0];
        m_submitLayers[m_submitLayer][1] = perEye[1];
        m_submitLayer++;
    }
    else {
        Warn("Too many layers submitted!\n");
    }

    if (Settings::Instance().m_DriverTestMode & 8) {
        // Crash test
        *(char*)0 = 1;
    }

    //CopyTexture();
}

In the above code snippet, I know m_poseHistory queues Poses. By the way, I cannot find any parts that performs reprojection using the pose history or m_framePoseRotation information. I'd appreciate it if anyone helps me to locate the implementation code part.

Thanks in advance.

[zarik5]

The ALVR server does not perform reprojection. we get the orientation of the first layer and assume all other layer have the same orientation (which is sometimes wrong and it's the cause of strange glitches especially during loading screens). For reprojection we leverage the client VR runtime, and it's enabled automatically by the client OS.

[leedc20]

Hi @zarik5, Thank you for your updating. Would you let me know what you mean by the client VR runtime? you mean SteamVR?

[zarik5]

No. I mean the VR runtime that powers the headset, and does not depend on the API used (VrApi or OpenXR) and it can be updated together with the firmware. It is also called "compositor", although it's a bit confusing because ALVR has a client compositor too (which does FFR but not reprojection).

[leedc20]

Okay. I'm going to implement own reprojection in my driver runtime (which was built using OpenVR) for my custom headset. Would you guide me how to implement that?

[zarik5]

You can write me on Discord since this is off topic. I'm zarik#6062

[leedc20]

Thank you @zarik5 . I have been away from my dev. I will contact you on Discord. Looking forward to getting your advice a lot.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[leedc20]

Hi @zarik5, I've sent you a connection on Discord. My discord id is leec#7849. Looking forward to getting your great help. Thanks in advance.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[yingtian991221]

Hi, I'm curious about your implementation of reprojecting. As I'm using Oculus Quest, I wonder how timewarp works without VrApi. Could you give me some hints? I also sent a friend request to you on Discord and my id is Yingtian#9790