IVRSystem::GetProjectionMatrix returns a wrong matrix

[risa2000]

When debugging some projection issues in a Unity game I went all the way down to OpenVR API and found this strange (and I assume wrong) behavior of IVRSystem::GetProjectionMatrix: (I am using Pimax 5k+ HMD and pyopenvr to run the simple queries.)

1) Unity has a function Camera.GetStereoProjectionMatrix, which returns this value for the left eye:

      [[ 0.64759,  0.     , -0.12824,  0.     ],
       [ 0.     ,  0.7875 ,  0.     ,  0.     ],
       [ 0.     ,  0.     , -1.0002 , -0.20002],
       [ 0.     ,  0.     , -1.     ,  0.     ]]

The doc says (https://docs.unity3d.com/ScriptReference/Camera.GetStereoProjectionMatrix.html) that the value is usually provided directly by VR SDK (unless explicitly changed, which it was not).

2) OpenVR has a function IVRSystem::GetProjectionMatrix, which, when called on the same HMD, using the same near and far clipping planes as Unity does (near=0.1, far=1000.1) returns for the left eye this:

[[ 0.64759356  0.         -0.12823947  0.        ]
 [ 0.          0.78749996  0.          0.        ]
 [ 0.          0.         -1.0001     -0.10001001]
 [ 0.          0.         -1.          0.        ]]

Which is not the same matrix. Then I noticed that OpenVR has also a function which returns the "raw" values reported directly by the headset IVRSystem::GetProjectionRaw. Running this function for the left eye gives:

tan_left=-1.742203, tan_right=1.346154, tan_bottom=-1.269841, tan_top=1.269841

When I run my function, which builds the projection matrix with the formulas described for example here (http://www.songho.ca/opengl/gl_projectionmatrix.html), and use "raw" values obtained above and add near and far clipping planes defined by Unity, I get the same matrix as Unity has:

[[ 0.64759358  0.         -0.12823948  0.        ]
 [ 0.          0.78749997  0.          0.        ]
 [ 0.          0.         -1.0002     -0.20002   ]
 [ 0.          0.         -1.          0.        ]]

On the other hand, if I take the result of IVRSystem::GetProjectionMatrix mentioned above and compute corresponding values for left, right, bottom, top, near and far coordinates, I get these values:

l=-0.087115, r=0.067311, b=-0.063495, t=0.063495, n=0.050003, f=999.934148

Which do not correspond to the "raw" values for the near and far clipping planes.

Further, the doc page for IVRSystem::GetProjectionRaw (https://github.com/ValveSoftware/openvr/wiki/IVRSystem::GetProjectionRaw) suggests a function ComposeProjection for calculating the projection matrix from the raw values, which, when implemented, gives the same wrong result as the current API.

It seems that there is bug in current implementation of IVRSystem::GetProjectionMatrix. The correct one (using ComposeProjection as an example) should be:

void ComposeProjection(float fLeft, float fRight, float fTop, float fBottom, float zNear, float zFar,  HmdMatrix44_t *pmProj )
{
    float idx = 1.0f / (fRight - fLeft);
    float idy = 1.0f / (fBottom - fTop);
    float idz = 1.0f / (zFar - zNear);
    float sx = fRight + fLeft;
    float sy = fBottom + fTop;

    float (*p)[4] = pmProj->m;
    p[0][0] = 2*idx; p[0][1] = 0;     p[0][2] = sx*idx;    p[0][3] = 0;
    p[1][0] = 0;     p[1][1] = 2*idy; p[1][2] = sy*idy;    p[1][3] = 0;
    p[2][0] = 0;     p[2][1] = 0;     p[2][2] = -(zFar + zNear)*idz; p[2][3] = -2*zFar*zNear*idz;
    p[3][0] = 0;     p[3][1] = 0;     p[3][2] = -1.0f;     p[3][3] = 0;
}

[Scawen]

Hi risa2000, I realise your post is about the values in the 3rd row of the matrix, related to zNear and zFar, but I noticed something I would like to check with you, although what I am saying doesn't affect the substance of your report.

You said, as the result of GetProjectionRaw for your headset: tan_left=-1.742203, tan_right=1.346154, tan_bottom=-1.269841, tan_top=1.269841

But I am wondering if you have swapped the bottom and top values. Why I ask: there is a known issue in which the top and bottom results are swapped, which can cause a problem with headsets which have a different bottom and top tan, if the programmer isn't aware of the bug. Top results are (wrongly) returned as negative values while bottom results are positive. This inverted result seems to be subsequently cancelled out by the ComposeProjection function.

My report of the known issue, which links to two other reports: https://github.com/ValveSoftware/openvr/issues/1050 For example, with a Vive DVT, the results from GetProjectionRaw are: eye L: Left -1.391 Right 1.254 Top -1.473 Bottom 1.466 eye R: Left -1.244 Right 1.402 Top -1.474 Bottom 1.463

[risa2000]

@Scawen you are right, I noticed that the function declaration has top and bottom "swapped" in the header: void GetProjectionRaw( Hmd_Eye eEye, float *pfLeft, float *pfRight, float *pfTop, float *pfBottom ) I wrote swapped because generally it is declared in the opposite order, but since I was using pyopenvr wrapper and the values seemed to be right the other way around I assumed the Python function was returning it in the "correct" order.

And now, when looking at ComposeProjection I realized that I also "autocorrected" the "correction" you mention, so I basically implemented: float idy = 1.0f / (fTop - fBottom);

Anyway, it seems that the doc probably needs to get some review to admit all the unexpected behavior in the first place, and then possibly correct the issue I reported.