Open haugeSander opened 4 months ago
We could also reference the OpenGL docs: https://registry.khronos.org/OpenGL-Refpages/gl2.1/xhtml/glFrustum.xml
Skewed frustum matrices:
The paper is primary for 16 screen displays, but it will also work with a single screen.
N and F values has to be set based on the scene I think. Where N is a relative small value. and F is the distance to the object that is the longest distance away.
In C++, matrices are either al list or 2d array. The only problem what we need to solve is how to apply this to the camera.
Very good article on the effect of applying a costume projection matrix in Unreal Engine 5: https://forums.unrealengine.com/t/camera-projection-matrices/9408
Done in Unity: https://edom18.medium.com/implementation-of-generalized-perspective-projection-on-the-unity-c9472a94f083
Might be useful: https://speckle.systems/tutorials/unreal-developing-custom-conversion-logic/
Also to explain coordinate systems:
Useful function to get screen size.
GetPixelSizeOfScreen ( float& Width, float& Height, UCanvas* Canvas, int32 LocalPlayerIndex )
#pragma once
#ifndef PROJECTION_H
#define PROJECTION_H
#include "Math/Vector.h"
class ProjectionCalculation
{
public:
ProjectionCalculation();
~ProjectionCalculation();
FMatrix ProjectionCalculation::get_projection_matrix(const FVector& pa, const FVector& pb, const FVector& pc, const FVector& pe, float near_clip, float far_clip);
private:
};
#endif
#include "Projection.h"
#include "Math/Matrix.h"
ProjectionCalculation::ProjectionCalculation() {
};
ProjectionCalculation::~ProjectionCalculation() {};
FMatrix ProjectionCalculation::get_projection_matrix(const FVector& pa,
const FVector& pb,
const FVector& pc,
const FVector& pe,
float near_clip,
float far_clip) {
// Compute the screen's orthonormal basis vectors
FVector vr = (pb - pa).GetSafeNormal();
FVector vu = (pc - pa).GetSafeNormal();
FVector vn = FVector::CrossProduct(vr, vu).GetSafeNormal();
// Compute vectors from pe to each of the screen's corners
FVector va = pa - pe;
FVector vb = pb - pe;
FVector vc = pc - pe;
// Compute the distance from the eye to screen plane
float d = -FVector::DotProduct(va, vn);
// Compute the extents of the perpendicular projection
float l = FVector::DotProduct(vr, va) * near_clip / d;
float r = FVector::DotProduct(vr, vb) * near_clip / d;
float b = FVector::DotProduct(vu, va) * near_clip / d;
float t = FVector::DotProduct(vu, vc) * near_clip / d;
// Construct the projection matrix
FMatrix projectionMatrix = FMatrix();
projectionMatrix.M[0][0] = 2.0f * near_clip / (r - l);
projectionMatrix.M[1][1] = 2.0f * near_clip / (t - b);
projectionMatrix.M[2][2] = -(far_clip + near_clip) / (far_clip - near_clip);
projectionMatrix.M[2][3] = -1.0f;
projectionMatrix.M[3][2] = -(2.0f * far_clip * near_clip) / (far_clip - near_clip);
projectionMatrix.M[0][2] = (r + l) / (r - l);
projectionMatrix.M[1][2] = (t + b) / (t - b);
projectionMatrix.M[3][3] = 0.0f;
// Construct the rotation matrix from the basis vectors
FMatrix rotationMatrix = FMatrix();
rotationMatrix.M[0][0] = vr.X; rotationMatrix.M[1][0] = vr.Y; rotationMatrix.M[2][0] = vr.Z;
rotationMatrix.M[0][1] = vu.X; rotationMatrix.M[1][1] = vu.Y; rotationMatrix.M[2][1] = vu.Z;
rotationMatrix.M[0][2] = vn.X; rotationMatrix.M[1][2] = vn.Y; rotationMatrix.M[2][2] = vn.Z;
// Combine the matrices
FMatrix finalMatrix = rotationMatrix * projectionMatrix;
return finalMatrix;
}
FVector pa(0, 0, 0), pb(1, 0, 0), pc(0, 1, 0), pe(0.5, 0.5, 1);
Download this: https://github.com/aptas/off-axis-projection-unity, and see if the effect is as good as it seems.
Found in this article: https://medium.com/try-creative-tech/off-axis-projection-in-unity-1572d826541e.
Notes on projection matrix in this presentation. Found lower down within the presentation https://courses.cs.washington.edu/courses/cse455/09wi/Lects/lect5.pdf#:~:text=URL%3A%20https%3A%2F%2Fcourses.cs.washington.edu%2Fcourses%2Fcse455%2F09wi%2FLects%2Flect5.pdf%0AVisible%3A%200%25%20
Done a few times before: https://github.com/topics/off-axis-projection
The original article, the wii remote: http://johnnylee.net/projects/wii/
This is for Unity but might be an idea: https://github.com/AmauryFauvel/Parallax
Found some more resource:
Unity Implementation:
Article on Stereo Rendering: https://paulbourke.net/stereographics/stereorender/
Here is a way to access the projection matrix:
Also recommended following API docs: https://dev.epicgames.com/documentation/en-us/unreal-engine/API/Runtime/Engine/Camera/UCameraComponent/GetCameraView?application_version=5.3
There is maybe something in this article: https://forums.unrealengine.com/t/howto-modify-the-projection-matrix/287457/15
Code from 2014 that should work with UE4:
public static Matrix4 UpdateFrustrum(float ScreenWidth,float ScreenHeight,Vector3 eyePosition)
{
float headX = eyePosition.X / ScreenHeight;
float headY = eyePosition.Y / ScreenHeight;
float headDist = eyePosition.Z / ScreenHeight;
float screenAspect = ScreenWidth / ScreenHeight;
float nearPlane = 1f;
Matrix4 resultM = Matrix4.CreatePerspectiveOffCenter( nearPlane*(-.5f * screenAspect - headX)/headDist,
nearPlane*(.5f * screenAspect - headX)/headDist,
nearPlane*(-.5f + headY)/headDist,
nearPlane*(.5f + headY)/headDist,
nearPlane, 1000);
return MagicFix (resultM);
}
private static Matrix4 MagicFix(Matrix4 matrix){
float[,] result = Matrix2Float (matrix);
//result [1, 2] = 0f;
result[2,3] = 1f;
result[2,2] = 0.0f;
result[3,0] = 0.0f;
result[3,1] = 0.0f;
result = ScaleMatrix(result, (1.0f / result [0, 0]));
result[3,2] = nearClipPlane;
return Float2Matrix (result);
}
People keep tacking about some sort of magic when it come to the translation of the matrix. Idk why.
Also some idea: https://www.reddit.com/r/unrealengine/comments/qcudtv/looks_like_hes_sitting_behind_the_screen_playing/
He achieved this by:
I made a class derived from FSceneViewExtensionBase to modify the OffCenterProjectionOffset
Here is the starting function I have been working on. People in the community is talking about a magic fix.
FMatrix AMovableCamera::UpdateFrustrum(float x, float y, float z)
{
float headX = x / (CX * 2);
float headY = y / (CY * 2);
float headDist = z;
float screenAspect = CX / CY;
float nearPlane = 1.0f;
FMatrix resultM = FMatrix::Identity;
// Populate the perspective matrix
resultM.M[0][0] = nearPlane / (-(.5f * screenAspect - headX) / headDist);
resultM.M[1][1] = nearPlane / ((.5f * screenAspect - headX) / headDist);
resultM.M[2][0] = (.5f - headY) / headDist;
resultM.M[2][1] = (.5f + headY) / headDist;
resultM.M[2][2] = 0.0f;
resultM.M[2][3] = 1.0f;
resultM.M[3][2] = nearPlane;
resultM.M[3][3] = 0.0f;
// Apply magic fix
float result[4][4];
for (int32 i = 0; i < 4; ++i)
{
for (int32 j = 0; j < 4; ++j)
{
result[i][j] = resultM.M[i][j];
}
}
// Apply fix
result[2][2] = 0.0f;
result[3][0] = 0.0f;
result[3][1] = 0.0f;
result[3][2] = nearPlane;
// Scale matrix
float scaleFactor = 1.0f / result[0][0];
for (int32 i = 0; i < 4; ++i)
{
for (int32 j = 0; j < 4; ++j)
{
result[i][j] *= scaleFactor;
}
}
// Convert result back to FMatrix
for (int32 i = 0; i < 4; ++i)
{
for (int32 j = 0; j < 4; ++j)
{
resultM.M[i][j] = result[i][j];
}
}
return resultM;
}
Currently experimenting with theis snippet of code:
FVector position(X,Y, Z);
FRotator rotator(0,0,0);
FVector scale(0, 0, 0);
FTransform cameratrans(rotator, position, scale);
CameraComponent->AddAdditiveOffset(cameratrans, 0);
What
A skewed frustum projection is used to correct the perspective in a 3D scene when the viewer is not directly in front of the screen. It adjusts the rendering so that the scene looks correct from the viewer's off-center position, creating an illusion of depth as if looking through a window. This is crucial for immersive experiences in virtual and augmented reality, where the user's viewpoint is constantly changing.
The user’s eye position may not be parallel to XYplane, which means we need to rotate the XY-plane to make it parallel to the user’s eye position.
Good source: Generalized Perspective Projection