Modify GetProjectionRaw from a console application (Anonymous VR)

[AnonymousVR-VR]

Modify GetProjectionRaw from a console application

Hi everyone, I'm trying to make a universal app for the MOD viewer community, in this case, I'm trying to modify the GetProjectionRaw values.

I am in the project of this modification of the FOV from 90ºFOV to 135ºFOV, modifying the angles of the screens, it is possible to achieve it.

I have used a VRTEK WVR3 Viewer that has a simple 3DOF IMU and I have been able to make a new controller using the OpenVR API, but this is not valid to make a universal solution for all viewers on the market.

You can see the project here:

https://www.realovirtual.com/foro/topic/51245/mod-visor-vr-los-90o-fov-135o-fov-anonymous-vr

The problem is, it's really hard to make new controllers for viewers, so the easiest way is to modify the GetProjectionRaw parameters externally.

I have been able to make a program where I can read the GetProjectionRaw parameters, but I don't know how to go about changing the Projection parameters.

How should one go about changing the parameters of GetProjectionRaw?

Can someone show some code example?

I can only read parameters, but not change them, I also have to say that I have hardly any experience in programming, I am a particular VR user, who tries to do things.

// Anonymous VR - MOD VISOR VR 90ºFOV @ 135ºFOV.

#include <openvr.h>

int main(int argc, char* argv[])                                                    
{
    vr::EVRInitError eError = vr::VRInitError_None;
    auto VRSystem = vr::VR_Init(&eError, vr::VRApplication_Other);

    float left, right, top, bottom;
    VRSystem->GetProjectionRaw(vr::Eye_Left, &left, &right, &top, &bottom);
    printf("Ojo Izq:  %8.5f, %8.5f, %8.5f, %8.5f\n", left, right, top, bottom);

    VRSystem->GetProjectionRaw(vr::Eye_Right, &left, &right, &top, &bottom);
    printf("Ojo Der: %8.5f, %8.5f, %8.5f, %8.5f\n", left, right, top, bottom);

    getchar();
    return 0;
}

As you can see, reading the parameters is very very simple. Does anyone know how to modify them in the same way?

Thanks for the collaboration. Anonymous VR.

[Rectus]

It is not possible to change the values directly from the application-side API.

Most likely the only way to make an universal application is by hooking functions in the driver interface, and inserting your application between the driver and SteamVR, intercepting the calls between them. This method is not officially supported by Valve, likely requires much work and programming expertise to implement, as well as requiring maintenance every time the interface changes. There are several applications like OpenVR Space Calibrator that use hooks to modify values supplied by drivers.

[AnonymousVR-VR]

It is not possible to change the values directly from the application-side API.

Hello Rectus in openvr.h you can find the following methods, which can be used in your application.

// Display Methods
/** The projection matrix for the specified eye */
virtual HmdMatrix44_t GetProjectionMatrix( EVREye eEye, float fNearZ, float fFarZ ) = 0;

/** The components necessary to build your own projection matrix in case your application is doing something
 fancy like infinite Z */
virtual void GetProjectionRaw( EVREye eEye, float *pfLeft, float *pfRight, float *pfTop, float *pfBottom ) = 0;

The components necessary to build your own projection matrix in case your application is doing something fancy like infinite Z

Openvr.h Display Methods

These are some of the mentioned methods to be used in applications such as OpenGL and DX12, there are some more, you can find them in Openvr.h which differs from openvr_driver.h, which includes its own methods for drivers.

[Rectus]

Correct me if I'm wrong, but as I understand it you're trying to customize the optics of a VR headset, and need a way of change the projection parameters for all user applications.

The methods in openvr.h are meant for user applications (like games) to retrieve the matrix parameters and use them for rendering. The applications can retrieve them either as a complete projection matrix, or the FoV parameters for constructing the matrix. Since they are for user applications, they do not support writing the values back to SteamVR.

[AnonymousVR-VR]

Correct me if I'm wrong, but as I understand it you're trying to customize the optics of a VR headset, and need a way of change the projection parameters for all user applications.

Yes, that is correct.

What is being tried to do is an injector or application that can change the parameters when SteamVR has already loaded the user controller.

Due to the changing angles of the screens, you have to wrap the GetProjectionRaw data externally somehow, or find another way to do it.

There are applications that make modifications to create other lens effects or reprojections.

Unfortunately the drivers or driver .dll files do not have all the GetProjectionRaw functions compiled into them, so it is not possible to directly modify the .dll to modify the parameters.

Here an example using Ghidra:

Driver with the full functions of GetProjectionRaw for the 2 eyes.

Controller with the standard functions in GetProjectionRaw for 1 eye only.

I have tried to find solutions for this problem.

I don't have enough knowledge to make a code from scratch, I need examples to do things and unfortunately there isn't much on how to make configuration changes from an application.

As I have commented, it is possible to make a new controller and add the GetProjectionRaw functions as you can see in the images I have posted, but it is very, very difficult to provide the new controller with all the requirements to be a good native controller.

In addition to adding the correction functions for each eye individually, it is necessary to add the functions of lens correction, aberrations, tracking and other functions.

Making a controller for an HMD VR viewer is not an easy task... you have to try to do something externally.

I appreciate your interest in this topic and your help is appreciated, thank you very much.

[Rectus]

Thanks, that makes it clear. Still, the only option I can see that would work is hooking the driver.

[AnonymousVR-VR]

ChatGPT 1ª solutión: SetProjectionRaw

#include <openvr.h>
int main(int argc, char* argv[])                                                    
{
    vr::EVRInitError eError = vr::VRInitError_None;
    auto VRSystem = vr::VR_Init(&eError, vr::VRApplication_Other);

    float left, right, top, bottom;
    // Nuevos valores de proyección para el ojo izquierdo
    left = -0.5f;
    right = 0.5f;
    top = 0.5f;
    bottom = -0.5f;
    VRSystem->SetProjectionRaw(vr::Eye_Left, left, right, top, bottom);

    // Nuevos valores de proyección para el ojo derecho
    left = -0.5f;
    right = 0.5f;
    top = 0.5f;
    bottom = -0.5f;
    VRSystem->SetProjectionRaw(vr::Eye_Right, left, right, top, bottom);

        // VRSystem->ApplyTransform(NULL);   // Si los datos no son actualizados

    getchar();
    return 0;
}

[AnonymousVR-VR]

ChatGPT 2ª solutión: HmdMatrix44_t

#include <openvr.h>
#include <iostream>

int main(int argc, char* argv[])
{
    vr::EVRInitError eError = vr::VRInitError_None;
    auto VRSystem = vr::VR_Init(&eError, vr::VRApplication_Other);

    if (eError != vr::VRInitError_None)
    {
        std::cerr << "Error al inicializar OpenVR: " << vr::VR_GetVRInitErrorAsEnglishDescription(eError) << std::endl;
        return 1;
    }

    vr::HmdMatrix44_t matLeft = VRSystem->GetProjectionMatrix(vr::Eye_Left, 0.1f, 100.0f);
    vr::HmdMatrix44_t matRight = VRSystem->GetProjectionMatrix(vr::Eye_Right, 0.1f, 100.0f);

    // Modifica las matrices aquí si es necesario.
    matLeft.m[0][0] = 1.0f;
    matRight.m[0][0] = 1.0f;

    // Usa las matrices modificadas para dibujar el mundo VR.

    vr::VR_Shutdown();
    return 0;
}

[AnonymousVR-VR]

ChatGPT 3ª solutión:GetProjectionMatrix

#include <openvr.h>
#include <math.h>
#include <iostream>

int main(int argc, char* argv[])
{
    vr::EVRInitError eError = vr::VRInitError_None;
    vr::IVRSystem* VRSystem = vr::VR_Init(&eError, vr::VRApplication_Other);

    if (eError != vr::VRInitError_None)
    {
        std::cerr << "Error al inicializar OpenVR: " << vr::VR_GetVRInitErrorAsEnglishDescription(eError) << std::endl;
        return 1;
    }

    // Convertir los grados a radianes
    const float theta = 5.0f * (M_PI / 180.0f);
    const float cosTheta = cos(theta);
    const float sinTheta = sin(theta);

    // Crear la matriz de rotación
    vr::HmdMatrix34_t matRotation;
    matRotation.m[0][0] = cosTheta;
    matRotation.m[0][1] = -sinTheta;
    matRotation.m[1][0] = sinTheta;
    matRotation.m[1][1] = cosTheta;
    matRotation.m[2][2] = 1.0f;

    // Leer la matriz de proyección original
    vr::HmdMatrix44_t matProjection = VRSystem->GetProjectionMatrix(vr::Eye_Right, 0.1f, 100.0f);

    // Aplicar la rotación a la matriz de proyección
    for (int i = 0; i < 3; i++)
    {
        for (int j = 0; j < 4; j++)
        {
            float sum = 0.0f;
            for (int k = 0; k < 3; k++)
            {
                sum += matRotation.m[i][k] * matProjection.m[k][j];
            }
            matProjection.m[i][j] = sum;
        }
    }

    // Asignar la matriz de proyección modificada
    VRSystem->SetProjectionMatrix(vr::Eye_Right, matProjection);

    std::cout << "La matriz de proyección se ha rotado 5 grados a la derecha." << std::endl;

    return 0;
}

[AnonymousVR-VR]

ChatGPT 4ª solutión: ApplyTransform

#include <openvr.h>
#include <cmath>

int main(int argc, char* argv[])
{
    // Inicializar la API de OpenVR
    vr::EVRInitError eError = vr::VRInitError_None;
    auto VRSystem = vr::VR_Init(&eError, vr::VRApplication_Other);

    // Angulo de rotacion deseado en grados
    float rotationAngle = 5.0f;

    // Convertir el angulo a radianes
    float rotationAngleRad = rotationAngle * (M_PI / 180.0f);

    // Modificar los valores de proyeccion para el ojo izquierdo
    float left, right, top, bottom;
    VRSystem->GetProjectionRaw(vr::Eye_Left, &left, &right, &top, &bottom);

    // Calcular la matriz de proyeccion rotada
    vr::HmdMatrix34_t matLeft;
    matLeft.m[0][0] = cos(rotationAngleRad);
    matLeft.m[0][2] = -sin(rotationAngleRad);
    matLeft.m[1][1] = 1.0f;
    matLeft.m[2][0] = sin(rotationAngleRad);
    matLeft.m[2][2] = cos(rotationAngleRad);

    // Aplicar la matriz de proyeccion rotada
    VRSystem->ApplyTransform(&matLeft, vr::TrackingUniverseSeated, vr::EVRTrackedDeviceClass_HMD, vr::k_unTrackedDeviceIndex_Hmd);

    // Modificar los valores de proyeccion para el ojo derecho
    VRSystem->GetProjectionRaw(vr::Eye_Right, &left, &right, &top, &bottom);

    // Calcular la matriz de proyeccion rotada
    vr::HmdMatrix34_t matRight;
    matRight.m[0][0] = cos(rotationAngleRad);
    matRight.m[0][2] = -sin(rotationAngleRad);
    matRight.m[1][1] = 1.0f;
    matRight.m[2][0] = sin(rotationAngleRad);

    // Aplicar la matriz de proyeccion rotada
    VRSystem->ApplyTransform(&matRight, vr::TrackingUniverseSeated, vr::EVRTrackedDeviceClass_HMD, vr::k_unTrackedDeviceIndex_Hmd);

   std::cout << "La matriz de proyección se ha rotado 5 grados a la derecha." << std::endl;
    getchar();
    return 0;
}

[jerrypai025009]

Hello @AnonymousVR-VR, @Rectus, I am working on similar topic but using Unity editor with steamVR plugin. Since I'm also looking for method to modify the raw projection matrix but I can't get there. Are there any suggestions or references? Thank you.

[Rectus]

Hello @AnonymousVR-VR, @Rectus, I am working on similar topic but using Unity editor with steamVR plugin. Since I'm also looking for method to modify the raw projection matrix but I can't get there. Are there any suggestions or references? Thank you.

I have no experience with the Unity plugins, but both of them seem to be open-source. https://github.com/ValveSoftware/steamvr_unity_plugin https://github.com/ValveSoftware/unity-xr-plugin/

If this is for your own Unity application, you would probably have to modify either plugin, find where it calls IVRSystem::GetProjectionMatrix() or IVRSystem::GetProjectionRaw() and modify the output.