Raycasting transformation

[mfobert]

First of all - thank you for the library. It has saved me a lot of time and effort.

---

Video context: red sphere is mouse position set with setCoords (accurate), buildings are highlighted red when raycaster thinks they are intersected.

threebox transform.webm

Given a tb instance, I have manually added some lights (although this issue persists when only using defaultLights) and buildings.

Initialisation:

tb = new Threebox(map, map.getCanvas().getContext('webgl'), {
    defaultLights: false,
    enableSelectingObjects:true
});
window.tb = tb;
tb.renderer.shadowMap.enabled = true;
tbAmbientLight()
tbDirectionalLight()

map.addLayer({
    id: 'custom_layer',
    type: 'custom',
    renderingMode: '3d',
    onAdd: function (map, gl) {
        tbGroundClippingPlane()

    },
    render: function (gl, matrix) {
        tbBuildings()
        tb.update(); 
    }
});

Buildings/extrusions:

...
 const tb = window.tb;
    let s = tb.projectedUnitsPerMeter(tb.map.getCenter().lat);
    const buildingFeatures = tb.map.queryRenderedFeatures({
                layers: ['building'],
            });

    for (const feature of buildingFeatures) {
        const material1 = new THREE.MeshStandardMaterial({ color: "#ffffff", wireframe: false });
        const material2 = new THREE.MeshStandardMaterial({ color: "#ffffff", wireframe: false });
        if (feature.properties.extrude == "false"){ feature.properties.height = 20} 

        let extrusion = tb.extrusion({
            coordinates: feature.geometry.coordinates,
            geometryOptions: { 
                bevelEnabled: false, 
                depth: feature.properties.height*s ,
                  },
            materials: [material1, material2],
        });

        extrusion.model.castShadow = true;
        extrusion.model.receiveShadow = true;
        extrusion.model.onIntersect = function (intersect) {
            console.log("intersected building");
            extrusion.model.material = new THREE.MeshStandardMaterial({ color: "#ff0000", wireframe: false });
            tb.repaint();
        };
}
...

On mouse-move, I am setting a raycaster and trying to get extrusion intersections. I also tried to use tb.queryRenderedFeatures(point), but am receiving the same objects as with this raycaster.

const raycaster = new THREE.Raycaster();
let mousePos = new THREE.SphereGeometry(1, 32, 32);
const pointer = new THREE.Vector2();

map.on('mousemove', function(e){
    pointer.x = ( e.point.x / map._container.clientWidth ) * 2 - 1;
    pointer.y = - ( e.point.y / map._container.clientHeight ) * 2 + 1;
        mousePos.setCoords([e.lngLat.lng, e.lngLat.lat, 0]);
    raycaster.setFromCamera( pointer, tb.camera );
    const intersects = raycaster.intersectObjects( tb.scene.children );
    for ( let i = 0; i < intersects.length; i ++ ) {
        if (intersects[ i ].object.onIntersect){
            intersects[ i ].object.onIntersect()
        }
    }

The problem I am seeing is that the raycasting transformation seems to be scaled differently from what is actually rendered. Extrusions in the center of the canvas have accurate intersection events, but as the mouse moves outwards, the transformation becomes less accurate. The sphere manipulated with setCoords() is however accurate and follows the mouse nicely.

I have tried various methods to get the camera updated and project the point differently with no success. What am I missing here?

-- Also might be relevant, but maybe not. When scaling the browser, the extrusions are not updated as expected:

threebox scale.webm

Thanks!

[jeffmacinnes]

Hi @mfobert - have you had any luck finding a workaround for this? I'm facing a very similar issue. The mouseover detection is reasonably accurate in the center of the container, but the accuracy falls off as you move towards the periphery. The degree of error increases as it gets farther away from the center.

Thanks!!

https://user-images.githubusercontent.com/14912413/227114701-dd7a0685-0596-452e-9a2b-99dc1bd414d8.mp4

[mfobert]

Hi @mfobert - have you had any luck finding a workaround for this? I'm facing a very similar issue. The mouseover detection is reasonably accurate in the center of the container, but the accuracy falls off as you move towards the periphery. The degree of error increases as it gets farther away from the center.

Thanks!!

Hey @jeffmacinnes - I didn't have time to continue investigating how to get the ray-casting transformations aligned properly. The work around I used is to create a Threebox object that follows the mouse and threshold objects based on distance to the object's world coords. I could see this becoming problematic in your case as you have such high buildings and I am setting z=0 as it works well for my application. Still would be interested in a solution to this.

By the way the resizing issue I mentioned was unrelated!

[jeffmacinnes]

Thanks @mfobert - I did some more digging on this over the last couple of days. I don't have a satisfying answer to what's going wrong, but it does seem to be tied to changes on the Mapbox side. I had been using mapbox-gl v2.12, but then downgraded to v2.10 and all of the raycasting transformations were properly aligned.

This is consistent with another issue #376 that noticed (possibly related) bugs in Mapbox v2.11 as well.

Hope that fixes your issue too!

[GregorioF2]

I'm fighting a similar issue currently. In my case, the raycasting is working properly, but rather I find issues when resizing the map. It seems that during the raycasting action, it preserves the relative position of the objects on the scene, to where they were before the resize of the map was done.

So in my case, it seems that is more accurate towards the center because the relative position of the objects near the center is more similar to what it was before the transformation, to the ones on the edges.

https://user-images.githubusercontent.com/21204940/229957491-9a8361e4-9a9f-4a1f-9665-3426bafab385.mp4

This video was done with Mapbox v2.10.0. If anyone has any idea on what could be causing this, assistance would be much appreciated!

[GregorioF2]

I finally found a workaround for my problem executing on a map 'resize' event, a recreation of the camera. No the most elegant solution, but working for the moment. Basically executing:

this.map.on('resize', () => {
    const t = this.map.transform;
    this.fov = t.fov;
    const w = t.width;
    const h = t.height;
    const aspect = w / h;
    const orthographic = this.orthographic;
    if (orthographic) {
        this.map.transform.fov = 0;
        this.camera = new THREE.OrthographicCamera(w / - 2, w / 2, h / 2, h / - 2, 0.1, 1e21);
    } else {
        this.map.transform.fov = this.fov;
        this.camera = new THREE.PerspectiveCamera(this.fov, aspect, 2, 1e9);
    }
    this.camera.layers.enable(0);
    this.camera.layers.enable(1);
    this.cameraSync = new CameraSync(this.map, this.camera, this.world);
})