Raycast & Orthographic Camera

[jniac]

Hi Gordon,

First, i do really love your framework. Readable, straight to the point. For learning GL it's perfect!

Not really an issue, but a request. I like to use orthographic camera. But then raycasting is not working anymore. Since your code is very clear, i managed to get it work with two modifications :

First, Raycast.js, where origin and direction must be set in an other way:

    castMouse(camera, mouse = [0, 0]) {
        if (camera.type === 'orthographic') {
            // Set origin
            // Since camera is orthographic, origin is not the camera position
            const {left, right, bottom, top} = camera;
            const x = left + (right - left) * (mouse[0] * .5 + .5);
            const y = bottom + (top - bottom) * (mouse[1] * .5 + .5);
            this.origin.set(x, y, 0);
            this.origin.applyMatrix4(camera.worldMatrix);

            // Set direction
            // https://community.khronos.org/t/get-direction-from-transformation-matrix-or-quat/65502/2
            this.direction.x = -camera.worldMatrix[8];
            this.direction.y = -camera.worldMatrix[9];
            this.direction.z = -camera.worldMatrix[10];
        } else {
            // Set origin
            camera.worldMatrix.getTranslation(this.origin);

            // Set direction
            this.direction.set(mouse[0], mouse[1], 0.5);
            camera.unproject(this.direction);
            this.direction.sub(this.origin).normalize();
        }
    }

Then, since i'm trying to get back {left, right, bottom, top} values from camera, we need to store it right? so in Camera.js:

    orthographic({
        near = this.near,
        far = this.far,
        left = -1,
        right = 1,
        bottom = -1,
        top = 1,
    } = {}) {
+       Object.assign(this, {left, right, bottom, top});
        this.projectionMatrix.fromOrthogonal({left, right, bottom, top, near, far});
        this.type = 'orthographic';
        return this;
    }

Would you me to make a PR ?

Also, since you've been writing helper function as Mat4Func/getTranslation I was wondering if it would be relevant to have some:

• Mat4Func/getUp(out, mat)
• Mat4Func/getDown(out, mat)
• Mat4Func/getLeft(out, mat)
• Mat4Func/getRight(out, mat)
• Mat4Func/getBack(out, mat)
• Mat4Func/getFront(out, mat)

As an example Unity is providing such values (ex forward) from the Transform class. Maybe ThreeJS too.

This will allow to write the Raycast.castMouse method in a clearer way :

if (camera.type === 'orthographic') {
    // Set origin
    ...

    // Set direction
    camera.worldMatrix.getFront(this.direction);
}

What do you think about it ?

[jniac]

https://jniac.github.io/ogl/examples/?src=raycasting.html

[gordonnl]

Hey Joseph, thanks a lot for this, it's looking great.

I never got around to testing orthographic ray-casting, however I'm totally on board with everything you've described!

I actually think the Camera class could use a bit of a re-write that takes into account the camera type and storing the values...

Maybe let me have a pass at doing that first - keeping in mind what you've written above - then I'd love a PR for the Raycast and Matrix stuff!

Just as a note, the math classes are based from gl-matrix (with minor changes here and there).

[gordonnl]

Ok! I reworked the Camera class a bit in commit 1a104b1.

As well as being able to access the values for use in ray casting, storing properties should make cameras easier to use too in some cases! Especially in not needing to provide each of left, right, bottom, top to the .orthographic() method - only the values that have changed.

I also added a zoom property as per request in another issue: #36. This will affect your Raycast.castMouse() integration a tiny bit, as you'll need to divide the left, right, bottom, top by zoom to get the actual projected values.

Thanks again mate!

[jniac]

Hi Nathan, I pulled your commits, and i was about making a pull request when i ran into an expected issue. It concerns raycasting hit distance. I was implementing a kind of rollover / rollout system and needed to get the nearest object from camera. Everything was ok until i change the scale value of Mesh instance. By using the "local distance", computed from the "local" origin and direction of the ray, the current code assumes that the matrix would never be "stretched" (scale === [1,1,1]).

Without any changes the big yellow cube here (scale [5,5,5]) can lead to compute a smaller hit distance than the front normal cube (when hovering both). https://jniac.github.io/ogl/examples/?src=raycasting-debug.html

To fix that, we should convert the local hit point into world space, then compute a new distance from the origin. It implies a little more computations. But it seems to be worth the cost, and providing localPoint and (world space) point into hit object seems to make sense (before adding someday normal info?).

Let me know what would you like me to do (taking into account the scale and improving hit object, doing the PR, let you do the changes).

[jniac]

About the Matrix4 little helpers function, what name convention would you follow ?

• Left / Right / Up / Down / Front / Back
• Left / Right / Upward / Downward / Forward / Backward

[gordonnl]

Hey mate, great pickup for the distance transformation error!

I think your solution to convert the local hit point is perfect, please do add it to your PR! For the 'normal/uv/etc' info, I think that will be achievable with a barycentric hit-test (ray cast against each triangle). I haven't opted to doing that yet as I find box and sphere tests cover 90% of use cases.

For the naming convention (left/right/up/etc), I think the first option is best (omitting the '-ward')! I normally opt for shorter, and I don't think it can be confused with other meanings.

Thanks!

[jniac]

You're right, box + sphere cover 90% of use cases. I made a first PR with only the Raycast.js fixes.

[jniac]

Hi Nathan, I think i can achieve the normal hit computation. Once the bounds are hit, we go down to each triangles, loop over them to find the triangles that are hit by the ray. To compute the point we can use barycentric coordinates, you're right. Once the ray is converted to the triangle space, we can compute (u, v) and check that (u, v) is within the triangle. I may have a try. Since i feel the need of better understanding 3D geometry it could be a good exercise. Have you planned something on your side ?

[gordonnl]

Hey mate, yeah that sounds right - you'd definitely want to test the bounds before starting to test against each triangle.

Here is the intersectTriangle math from Threejs, probably a great starting point.

As for the API, I think we could probably add an intersectTriangle method, and similar to intersectBounds we could add intersectMeshes, to indicate that it will test against the geometry itself, not just the bounds.

There's probably a way to merge those two, and have an argument to flag that you just want to check the bounds or something...

Let me know what you think!

[jniac]

Hi Nathan, Here is a draft: examples/raycasting-triangles.html

Please do not look at the code inside the html page, i've tried different options, copied & pasted some of your example, without cleaning anything... ok.

Here are the line that compute the triangle raycast. Not so much. I wrote it my style (no semicolon, leading blank line, no inline-if) because it's a draft, if there's a PR i will rewrite it in ogl actual style.

It's not exactly the same code than three.js which i do not really understand in the details, but we are trying to solve the same equation.

The one i use is O + k * D = A + ku * U + kv * V

For better performances, i prefer to dereference all the component (x, y, z) from the vector, and write directly dot and cross product rather than calling function (since calculations are not as dense as in the case of matrices). Generally speaking i do not think that seeking optimization in javascript is a primary goal, but here, since meshes can have thousands of triangles, i prefer to be careful.

If the code for raycast triangle seems ok, i still have bug with non-indexed geometry (sometimes hover produce flickering, triangles are not drawn under the mouse): examples/raycasting-triangles.html?fox=true.

I do not have time yet to understand why. A first bug came from null triangle than can be provided in models, so i wrote a check first. For non-indexed geometries, every 3 vertices from the position attributes represent a triangle right?

Also, raycast may have some option (backface culling, distance max), actually i only return the closest triangles, but it may be useful to have all the triangles (in case of non-convex geometries). And also actually the test is made mesh by mesh, but may be done from any node (eg: scene) by walking through every child.

At last, is there actually a way to display debug lines / dot / vectors? It could be useful to display pivot from any Transform, normal from raycast etc. What could be the best option to achieve that do you think?

Many questions. Not sure the current issue is really concerned. Should we move the discussion elsewhere?