RayCaster: Allow object tests to be more easily extended.

[BlakeHancock]

Right now the only way to tell the RayCaster to include an object or not is with setting layers, and it is limited to checking any one of the RayCaster's layers is included in the objects layers.

Because this check exists outside the object in a separate function, in order to override this behavior, you need to needlessly override the RayCaster's intersectObject and intersectObjects functions, instead of just the outside intersectObject function.

Possible Solutions:

1. Move the outside intersectObject function into the RayCaster object so that only it needs to be extended. (Maybe also include a second function for just the if ( object.layers.test( raycaster.layers ) ) { part.)
2. Add some sort of specifiable test mode to the Layers object to allow for all, exact, and none alternative comparison modes to take advantage of the bit masking.
3. Allow passing in a custom test object/function into intersectObject and intersectObjects.

Example Current Override It requires a lot of duplicate code to get the desired result.

import {
    RayCaster as ThreeRayCaster
} from 'three'

function ascSort(a, b) {
    return (a.distance - b.distance)
}

export default class extends ThreeRayCaster {
    constructor (origin, direction, near, far) {
        super(origin, direction, near, far)

        this._testMode = 'all'
    }

    get testMode () { return this._testMode }
    set testMode (value) { this._testMode = value }

    _intersectObject (object, intersects, recursive) {
        if (this._test(object)) {
            object.raycast(this, intersects)
        }

        if (recursive === true) {
            const children = object.children

            for (let i = 0, l = children.length; i < l; ++i) {
                this._intersectObject(children[i], intersects, true)
            }
        }
    }

    _test (object) {
        switch (this.testMode) {
            case 'any':
                return object.layers.test(this.layers)
            case 'none':
                return !object.layers.test(this.layers)
            case 'exact':
                return (object.layers.mask === this.layers.mask)
            case 'all':
            default:
                return ((object.layers.mask & this.layers.mask) === this.layers.mask)
        }
    }

    intersectObject (object, recursive, optionalTarget) {
        const intersects = optionalTarget || []

        this._intersectObject(object, intersects, recursive)

        intersects.sort(ascSort)

        return intersects
    },

    intersectObjects (objects, recursive, optionalTarget) {
        const intersects = optionalTarget || []

        for (let i = 0, l = objects.length; i < l; i ++) {
            this._intersectObject(objects[i], intersects, recursive)
        }

        intersects.sort(ascSort)

        return intersects
    }
}

Potential Override If More Extendable

import {
    RayCaster as ThreeRayCaster
} from 'three'

export default class extends ThreeRayCaster {
    constructor (origin, direction, near, far) {
        super(origin, direction, near, far)

        this._testMode = 'all'
    }

    get testMode () { return this._testMode }
    set testMode (value) { this._testMode = value }

    _test (object) {
        switch (this.testMode) {
            case 'any':
                return object.layers.test(this.layers)
            case 'none':
                return !object.layers.test(this.layers)
            case 'exact':
                return (object.layers.mask === this.layers.mask)
            case 'all':
            default:
                return ((object.layers.mask & this.layers.mask) === this.layers.mask)
        }
    }
}

[Mugen87]

Can you please describe at least one use case that demonstrates why the existing layer tests in context of raycasting are insufficient? The engine does not enhance APIs without a compelling reason. If problems can be solved on application level in an adequate way, the API is usually not enhanced either.

[BlakeHancock]

I have my see through objects, such as a window, in a transparent layer and my non see through objects in an opaque layer. I also have an enabled layer. I would want to test against objects that are enabled but not see through.

I suppose maybe I am abusing object layers in a way it wasn't meant to be used. Even so, if the test function was overridable I would still be able to test against any number of object criteria not based on layers.

I could always pre-filter the objects I send to the RayCaster, it just seemed more efficient to just have to filter once.

[Mugen87]

I have my see through objects, such as a window, in a transparent layer and my non see through objects in an opaque layer. I also have an enabled layer. I would want to test against objects that are enabled but not see through.

Visibility and raycasting don't have to share common layers. You can put the camera in one layer, the raycaster in another one. By assigning the objects to both layers or only one of them, you can control both rendering and raycasting in an independent fashion.

[Mugen87]

I could always pre-filter the objects I send to the RayCaster, it just seemed more efficient to just have to filter once.

I don't understand how an enhanced Raycaster API allows you do to the filtering once. The tests have to be done one each raycasting procedure. I don't think there is a performance difference if you do this on application level by maintaining an array of objects that should be part of the raycasting process or by enhancing Raycaster.

[BlakeHancock]

I guess in the contexts of games, I was planning on having a normal sight RayCaster, an infrared RayCaster, an X-Ray RayCaster and using layers to denote which objects are penetrable by each. For the infrared one for an example, depth is a consideration, so if it passes through too many objects, it would stop.

Again I can certainly create a wrapper class or function that handles this logic and uses the default ray tracer, it just seemed convenient to do it like my example.

I'm admittedly pretty new to 3d stuff, so I don't know the best practices very well.

I simply thought it was too bad I couldn't extend this part of the RayTracer due to it existing outside the exported object.

[Mugen87]

Okay, understood. But since your requirements are quite application specific, there is not yet a compelling reason for changing/enhancing the API. We can reconsider this issue if more user are going to ask for similar features.