opengl-tutorials / ogl

http://www.opengl-tutorial.org
2.71k stars 926 forks source link

TestRayOBBIntersection issue #70

Open AzGilrock opened 6 years ago

AzGilrock commented 6 years ago

I've incorporated the TestRayOBBIntersection function from misc05_picking_custom.cpp into my own program and it was working great when I was in a perspective projection matrix but whenever I was in an ortho projection it was failing. I've done a lot of debugging and I believe there is a coding error in that function for the else part of each axis where it labels it as a "rare case". For ortho projections the ray_direction vector is always (0, 0, -1) which causes the dot product against every axis to be zero (i.e. the variable f = 0.0) causing it to take the "rare case" path for every axis. The problem is the variable "e" is being negated in the rare case if statement which causes a hit to fail.

This code snippet I attached below hard codes all the inputs to the functions to demonstrate the error. This should result in a direct hit to the center of the object. I had to modify 2 things in the TestRayOBBIntersection function to get this to result in a hit. 1) Modify the rare cases to change all occurrences of "-e" to "e". 2) Add a Boolean input to the function to indicate you are doing a 2D hit detection so it will skip the Z axis.

I've got it working in my program now but wanted to provide this feedback to help others.

int screenWidth = 1200;
int screenHeight = 700;
glm::vec3 ray_origin;
glm::vec3 ray_direction;

ProjectionMatrix = glm::ortho(0.0f, (float)screenWidth, 0.0f, (float)screenHeight, 1.0f, 100.0f);
ViewMatrix = glm::mat4(1.0f);

int mousex = 200;
int mousey = 100;

ScreenPosToWorldRay(
    mousex, screenHeight - mousey,
    screenWidth, screenHeight,
    ViewMatrix,
    ProjectionMatrix,
    ray_origin,
    ray_direction
);

glm::vec3 aabb_min = glm::vec3(0.0f, 0.0f, 0.0f);
glm::vec3 aabb_max = glm::vec3(5.0f, 5.0f, 0.0f);
glm::mat4 ModelMatrix = glm::translate(glm::mat4(1.0f), glm::vec3(197.5f, 602.5f, 0.0f));

float distance = 1000000000.0f;
float intersection_distance;
bool hit = false;

if (TestRayOBBIntersection(
    ray_origin,
    ray_direction,
    aabb_min,
    aabb_max,
    ModelMatrix,
    intersection_distance,
    true)
    ) {
    if (intersection_distance < distance) {
        distance = intersection_distance;
        hit = true;
    }
}