Threejs sprite is not work in mapbox

[Simon-wu]

mapbox-gl-js version: 3.4.3

browser: chrome 126

Steps to Trigger Behavior

I used the example Add a 3D model use threejs of official website to add sprite, but the display is not correct and I can't face the camera.

const customLayer = {
        id: '3d-model',
        type: 'custom',
        renderingMode: '3d',
        onAdd: function (map, gl) {
            this.camera = new THREE.Camera();
            this.scene = new THREE.Scene();

            // create two three.js lights to illuminate the model
            const directionalLight = new THREE.DirectionalLight(0xffffff);
            directionalLight.position.set(0, -70, 100).normalize();
            this.scene.add(directionalLight);

            const directionalLight2 = new THREE.DirectionalLight(0xffffff);
            directionalLight2.position.set(0, 70, 100).normalize();
            this.scene.add(directionalLight2);

            // use the three.js GLTF loader to add the 3D model to the three.js scene
            const loader = new THREE.GLTFLoader();
            loader.load(
                'https://docs.mapbox.com/mapbox-gl-js/assets/34M_17/34M_17.gltf',
                (gltf) => {
                    this.scene.add(gltf.scene);
                }
            );
            const texture = new THREE.TextureLoader().load('https://threejs.org/examples/textures/sprite0.png');
            let material = new THREE.SpriteMaterial({ map: texture ,side:THREE.DoubleSide});
            material.sizeAttenuation = false;
            let sprite = new THREE.Sprite(material);
            sprite.scale.set(100, 100, 1)
            this.scene.add(sprite);

            this.map = map;

            // use the Mapbox GL JS map canvas for three.js
            this.renderer = new THREE.WebGLRenderer({
                canvas: map.getCanvas(),
                context: gl,
                antialias: true
            });

            this.renderer.autoClear = false;
        },
        render: function (gl, matrix) {
            const rotationX = new THREE.Matrix4().makeRotationAxis(
                new THREE.Vector3(1, 0, 0),
                modelTransform.rotateX
            );
            const rotationY = new THREE.Matrix4().makeRotationAxis(
                new THREE.Vector3(0, 1, 0),
                modelTransform.rotateY
            );
            const rotationZ = new THREE.Matrix4().makeRotationAxis(
                new THREE.Vector3(0, 0, 1),
                modelTransform.rotateZ
            );

            const m = new THREE.Matrix4().fromArray(matrix);
            const l = new THREE.Matrix4()
                .makeTranslation(
                    modelTransform.translateX,
                    modelTransform.translateY,
                    modelTransform.translateZ
                )
                .scale(
                    new THREE.Vector3(
                        modelTransform.scale,
                        -modelTransform.scale,
                        modelTransform.scale
                    )
                )
                .multiply(rotationX)
                .multiply(rotationY)
                .multiply(rotationZ);

            this.camera.projectionMatrix = m.multiply(l);
            this.renderer.resetState();
            this.renderer.render(this.scene, this.camera);
            this.map.triggerRepaint();
        }```

 3. [https://codepen.io/simon-wu/pen/XWLbpoW](example code)

[alberto-taiuti-skydio]

This is because the way mapbox lets you interface with ThreeJS, changes the projection matrix from a "traditional" perspective matrix, to a perspective + (I guess) mercator projection + modelview.

This causes these lines in ThreeJS to not work anymore: https://github.com/mrdoob/three.js/blob/216398f77b220f2fa2277a9406aa316ad97fce16/src/renderers/shaders/ShaderLib/sprite.glsl.js#L23

Hence sprites can't be really used in a ThreeJS + Mapbox setup, where Mapbox and ThreeJS share the GL context and the camera is controlled by Mapbox, unfortunately.

If we could use Mapbox + ThreeJS without having to tangle the two matrices, we'd be golden. I tried to do this, but then you end up doing what ThreeBox did and have a separate Node that represents the world; also not ideal: https://github.com/peterqliu/threebox/blob/6e6a1ae3878c8557c20eb56ada53af843222f390/src/Camera/CameraSync.js#L116

Ignore my reply above; it applies only when you want constant screen-space sizing of your sprites. You'll want to look here: https://github.com/mapbox/mapbox-gl-js/issues/12513

[Simon-wu]

This is because the way mapbox lets you interface with ThreeJS, changes the projection matrix from a "traditional" perspective matrix, to a perspective + (I guess) mercator projection + modelview.

This causes these lines in ThreeJS to not work anymore: https://github.com/mrdoob/three.js/blob/216398f77b220f2fa2277a9406aa316ad97fce16/src/renderers/shaders/ShaderLib/sprite.glsl.js#L23

Hence sprites can't be really used in a ThreeJS + Mapbox setup, where Mapbox and ThreeJS share the GL context and the camera is controlled by Mapbox, unfortunately.

If we could use Mapbox + ThreeJS without having to tangle the two matrices, we'd be golden. I tried to do this, but then you end up doing what ThreeBox did and have a separate Node that represents the world; also not ideal: https://github.com/peterqliu/threebox/blob/6e6a1ae3878c8557c20eb56ada53af843222f390/src/Camera/CameraSync.js#L116

Ignore my reply above; it applies only when you want constant screen-space sizing of your sprites. You'll want to look here: #12513

Thank you very much.