Depth texture to a mesh

[shanerob1106]

I have a compute shader that uses the list of view space coordinates to build a mesh. For some reason the the mesh doesn't align with the environment it seems slightly shifted to the right and off-axis. I'm not too sure if it has to do with the ProjectionMatrix functions in the EnvironmentDepthAccess script mentioned in a previous issue?

[shanerob1106]

EDIT: By adjusted for FOV discrepancies between individual eyes (60°) and centre eye (90°) by:

• Modifying Calculate3DOFReprojection to use both eye and centre FOVs.
• Creating MakeCenterEyeAdjustmentMatrix for FOV-based scaling.
• Applying this adjustment in reprojection calculations.
• Updating the compute shader to properly use adjusted UV coordinates.

These changes corrected the rightward shift in in-depth mapping.

// Change to compute shader float SampleEnvironmentDepth(const float2 uv, const int slice) { const float4 reprojectedUV = mul(_EnvironmentDepthReprojectionMatrices[slice], float4(uv.x, uv.y, 0.0, 1.0));

// Adjust the UV coordinates based on the new reprojection matrix const float2 adjustedUV = reprojectedUV.xy / reprojectedUV.w;

const uint3 depthtextureuv = uint3(adjustedUV.x 512, adjustedUV.y 512, 0);

// ... (rest of the function remains the same) }

// Changes to EnvironmentDepthAccess public class EnvironmentDepthAccess : MonoBehaviour { // ... (existing code remains the same)

private void UpdateCurrentRenderingState()
{
    var leftEyeData = GetEnvironmentDepthFrameDesc(0);
    var rightEyeData = GetEnvironmentDepthFrameDesc(1);

    OVRPlugin.GetNodeFrustum2(OVRPlugin.Node.EyeLeft, out var leftEyeFrustrum);
    OVRPlugin.GetNodeFrustum2(OVRPlugin.Node.EyeRight, out var rightEyeFrustrum);

    // Get the center eye FOV
    OVRPlugin.GetNodeFrustum2(OVRPlugin.Node.EyeCenter, out var centerEyeFrustrum);

    _threeDofReprojectionMatrices[0] = Calculate3DOFReprojection(leftEyeData, leftEyeFrustrum.Fov, centerEyeFrustrum.Fov);
    _threeDofReprojectionMatrices[1] = Calculate3DOFReprojection(rightEyeData, rightEyeFrustrum.Fov, centerEyeFrustrum.Fov);

    // ... (rest of the method remains the same)
}

internal static Matrix4x4 Calculate3DOFReprojection(EnvironmentDepthFrameDesc frameDesc, Fovf eyeFov, Fovf centerFov)
{
    var screenCameraToScreenNormCoord = MakeUnprojectionMatrix(
        eyeFov.RightTan, eyeFov.LeftTan,
        eyeFov.UpTan, eyeFov.DownTan);

    var depthNormCoordToDepthCamera = MakeProjectionMatrix(
        frameDesc.fovRightAngle, frameDesc.fovLeftAngle,
        frameDesc.fovTopAngle, frameDesc.fovDownAngle);

    var depthCameraToScreenCamera = MakeScreenToDepthMatrix(frameDesc);

    // Adjust for center eye FOV
    var centerEyeAdjustment = MakeCenterEyeAdjustmentMatrix(eyeFov, centerFov);

    var screenToDepth = depthNormCoordToDepthCamera * depthCameraToScreenCamera *
                        centerEyeAdjustment * screenCameraToScreenNormCoord;

    return screenToDepth;
}

private static Matrix4x4 MakeCenterEyeAdjustmentMatrix(Fovf eyeFov, Fovf centerFov)
{
    float eyeHorizontalFov = Mathf.Atan(eyeFov.LeftTan) + Mathf.Atan(eyeFov.RightTan);
    float centerHorizontalFov = Mathf.Atan(centerFov.LeftTan) + Mathf.Atan(centerFov.RightTan);

    float scaleFactor = eyeHorizontalFov / centerHorizontalFov;

    var matrix = Matrix4x4.identity;
    matrix.m00 = scaleFactor;

    // Adjust the offset to center the scaled view
    matrix.m03 = (1 - scaleFactor) * 0.5f;

    return matrix;
}

// ... (rest of the class remains the same)

}

[TudorJude]

Thank you for providing this solution. I'm leaving this thread open for visibility.