Sometimes projection skips some triangles/faces

[trusktr]

I'm not sure why, but in some cases the projection will skip some triangles of a mesh, and also will render on the other side when it shouldn't. I'm not sure yet how exactly I trigger this case (I'm using my TS fork, but the code is the same, and I submitted the same bug fixes here as there).

Here is a video. Notice that half of the sphere never receives the projection, and also the projection seems to happen on both sides (unlike any of the examples).

https://user-images.githubusercontent.com/297678/167317886-a76715cf-0cfd-4370-bc97-3099a0fbb080.mp4

Do you have any ideas what could cause this?

---

I don't have a simple reproduction yet because I've made some custom elements for LUME that use ProjectedMaterial internally, allowing this to be possible (the above video is from the following HTML code):

<style>
    html,
    body {
        width: 100%;
        height: 100%;
        margin: 0;
        padding: 0;
        background: #222;
        touch-action: none;
    }
</style>

<script src="global.js"></script>

<lume-scene id="scene" perspective="800" webgl>
    <lume-point-light position="200 -200 200" intensity="0.6" color="white"></lume-point-light>
    <lume-ambient-light color="white" intensity="0.6"></lume-ambient-light>
    <lume-camera-rig active="false" initial-distance="500" max-distance="1500" min-distance="100"></lume-camera-rig>

    <lume-points
        id="model"
        has="ply-geometry"
        src="./models/shelby-gt350.ply"
        point-size="0.3"
        rotation="90 0 0"
        position="-250 0 0"
        size="0 0 0"
        scale="50 50 50"
    ></lume-points>

    <lume-node position="100 100 100" rotation="0 30 0">
        <lume-camera-rig active initial-distance="500" max-distance="1500" min-distance="100"></lume-camera-rig>
        <lume-sphere
            has="projected-material"
            projected-textures="#tex"
            size="50 50 50"
            color="white"
            mount-point="0.5 0.5 0.5"
        >
        </lume-sphere>
        <lume-node id="rotator" align-point="0.5 0.5 0.5">
            <lume-projected-texture
                id="tex"
                size="50 50 50"
                src="./images/monalisa-2.jpg"
                fitment="contain"
                position="0 0 100"
                mount-point="0.5 0.5 0.5"
            >
                <lume-plane
                    wireframe
                    size="1 1"
                    has="basic-material"
                    size-mode="proportional proportional"
                    color="skyblue"
                    sidedness="double"
                ></lume-plane>
            </lume-projected-texture>
        </lume-node>
    </lume-node>
</lume-scene>

<script>
    LUME.defineElements()

    rotator.rotation = (x, y, z) => [x, y + 0.3, z]
</script>

It seems to me, and I definitely don't doubt because I've encountered issues numerous times, that THREE.WebGLRenderer has state bugs that may affect this somehow.

How do I know Three.js has some fairly bad bugs? Well for example, and in my experience porting Three.js to WebAssembly (with AssemblyScript, a TypeScript to Wasm compiler) I have found, fixed, and added unit tests for multiple bugs that TypeScript would have otherwise caught, so I know from experience the code base is not in as good of a shape as it can be.

[trusktr]

Would you be willing to have a session to debug this? I added you on Twitter where we can arrange something over DM if you'd be interested.

I discovered that if I call material.project(mesh) too many times, I get the above problem. If I don't call it too many times, I get the expected result but only on first frame (unable to animate). Notice the projection does not go to both sides of the sphere, as we would expect:

https://user-images.githubusercontent.com/297678/167319121-88ab1bab-a980-4a9f-81cd-9d6c8358cfaa.mp4

[trusktr]

I'm not yet sure what's causing this on my end. LUME sets matrixAutoUpdate to false for all objects controlled by the custom elements, then it updates their world matrices at a certain time. Maybe this has to do with the issue.

In the meantime, I found the following temporary workaround on my end, in the GLSL code:

                    // bool isFacingProjector = dotProduct > 0.0000001;
                    bool isFacingProjector = true; // always apply it, for now

Somehow, the direction of the camera that ProjectedMaterial sees is different than what I actually have in my scene, which is why the above sphere front faces doesn't match with actual projection direction. Not sure what's up yet, but this workaround works for my current case.