Creating a screen to ray function

[ceptri]

I love the library and have been using it a lot of the past six months. I've built my matrices for open gl using:

mat4.perspective(manta.m_camera.m_fov, manta.m_camera.m_aspect, manta.m_camera.m_near, manta.m_camera.m_far, this.m_projectionMatrix); ... mat4.lookAt(vEye, vCameraTarget, vUp, cameraMatrix); ... mat4.multiply(this.m_projectionMatrix, this.m_cameraMatrix, this.m_projectionViewMatrix);

and

gl_Position = uWorldViewProjectionMatrix * vec4(aVertexPosition, 1.0);

in my shader.

But I am having absolutely no luck trying to build a ray from a screen coordinate. I tried:

    var invViewMatrix = mat4.create();
    mat4.inverse(manta.m_render.m_projectionViewMatrix, invViewMatrix);

    var dx = (x / (this.m_viewWidth / 2)) - 1;
    var dy = -((y / (this.m_viewHeight / 2)) - 1);

    var t1 = [dx, dy, -1, 1.0];
    var t2 = [dx, dy, 1, 1.0];

    mat4.multiplyVec4(invViewMatrix, t1);
    mat4.multiplyVec4(invViewMatrix, t2);

    var p1 = vec3.create( [ t1[0] / t1[3], t1[1] / t1[3], t1[2] / t1[3] ]);
    var p2 = vec3.create( [ t2[0] / t2[3], t2[1] / t2[3], t1[2] / t2[3] ]);

But this just seems to give me garbage results. Any ideas?

Thanks

[ceptri]

Ha ha! NEver mind. I found the mistake:

var p2 = vec3.create( [ t2[0] / t2[3], t2[1] / t2[3], t1[2] / t2[3] ]);

should be

var p2 = vec3.create( [ t2[0] / t2[3], t2[1] / t2[3], t2[2] / t2[3] ]);

and now it works great!

[toji]

Glad you figured out something that works for you!

Just wanted to mention, it sounds to me as if the functionality you're looking for may be covered by vec3.unproject, which takes screen space coordinates and determines where in 3D space they would reside with a given projection matrix.