Allow .obj & .gLTF material overwrite, or componentize envMap & sphericalEnvMap ?

[chrisnovello]

Adding an envMap to a gLTF in aframe is currently counterintuitive and involved, which is a blow to PBR workflows. Currently, one can't use the material component to overwrite attributes of materials imported from gLTF, or .obj with .mtl files.

I've been teaching with a-frame a lot, and this is becoming a common point of confusion with my students (they wonder why their low-roughness materials don't reflect anything, then learn about envMaps and try to add one by setting the attribute with materials, and are confused why it doesn't work)

Some ways this could be approached, not mutually exclusive:

1) Allow gLTF & obj+mtl material overwrites from the material component This seems most intuitive to me, is what I tried the first time, and is what some of my students have told me they tried. One would just selectively overwrite attributes (like envMap) using the material component. Curious to understand why this currently isn't allowed. Many circumstances where one might want to overwrite attributes of a material on a gLTF. I can see how it is unintuitive when a gltf gets used to store a whole scene.. but for envMap, a very common use is to apply the same envMap to all scene objects

2) Ship a-frame with an env-map component (w/sphericalEnvMap support) and have the material component continue to ignore attribute changes on gltf & obj+mtl This makes sense in some ways — the logic being that any attributes not imported via gLTF files can can then be set via aframe components. Is maybe premature given talk of this possible extension.

3) Leave as is. Expect users to write a component, traverse the three object graph, and modify materials, and/or let community components fill the need and hope users know (where) to look

gLTF-part is one partial answer, but this is still trouble when: a) using a .glb where it is harder to know the (sometimes auto-generated) keynames that correspond to gltf pieces b) a gltf contains a single highpoly object that is broken into tons of sub meshes (Unbound's export currently does this.. tho likely something of an edge case).

The problem is solved for me: I've made a simple component that can be attached to gLTFs to inject envMaps and spericalEnvMaps, and I'm thinking about making it work with all entities (to be attached to a scene and propagate to all models that have materials, for example)... I'll publish this shortly, but I wanted to bring it into discussion more broadly as I've seen several newcomers bump into it with the core library.

ps: <3 <3 <3 a-frame, one of the most important projects going right now

[chrisnovello]

Another option would be to build this into a-sky or an a-skybox entity. With an option to source external envmap images, or dynamically make a cubemap screenshot of the scene itself at the start of the sketch. Just thinking out loud here, curious what others think !

[donmccurdy]

Hi @paperkettle, thanks for the suggestion and student feedback 🙂 For one option there is a cube-env-map component in aframe-extras.

Allow gLTF & obj+mtl material overwrites from the material component

Somewhat related: https://github.com/aframevr/aframe/issues/3155. This behavior is what a lot of people expect, and it would be nice if it could just work, but it's a pretty large change. I would be curious to see if it can be added (either part of material or a new component) that doesn't require maintaining duplicate copies of the schema.

Is maybe premature given talk of this possible extension.

I'm not sure that would be ideal for A-Frame, as you wouldn't want to include a different envMap with everything model.

Another idea would be adding a property to the new background component:

<a-scene background="visible: false; addEnvMap: true; src: url(...);"></a-scene>

^Assumes we update background to support a texture.... right now it only allows color.

[PlumCantaloupe]

Yeah I agree on this. Would be nice if A-Frame had built in functionality for directly setting any material's envMap. Perhaps even better would be a global scene setting that traverses all models and sets it?

For GLTF I actually had to create another GLTF component to support it. An example below (probably rough as I am still wrapping my head around writing components - OT but would be nice if there was a "component extension model" :)

"use strict";

//!!TODO : figure out warning about "Timer GLTf exist / doesn't exist"
AFRAME.registerComponent('mdmu-gltf', {
    //dependencies: [''],
    schema: {
        // ... Define schema to pass properties from DOM to this component
        src:    {type: "selector",  default:null},
        envMap: {type: "selector",  default:null},
        color:  {type: "string",    default:"rgb(255,255,255)"}
    },
    multiple: false, //do not allow multiple instances of this component on this entity
    init: function() 
    {},
    update: function(oldData)  
    {
        let OContext    = this;
        let data        = OContext.data;
        let el          = OContext.el;
        OContext.loaded     = false;

        if (Object.keys(data).length === 0) { return; } // No need to update. as nothing here yet

        //reusable functions ...
        let changeColor = (color_rgb) => {
            MDMU.debugLog("COLOR change in mdmu-gltf " + color_rgb);

            let object  = el.getObject3D('mesh');

            if ( object !== undefined ) {
                let color   = color_rgb;

                object.traverse( function ( node ) {
                    if ( node.isMesh ) { 
                        if (node.material) {
                            node.material.color = new THREE.Color(color);
                            node.material.needsUpdate = true;
                        }
                    }
                });
            }
            else {
                console.log("Entity 3Dmesh doesn't exist yet");
            }
        };

        let changeEnvMap = (envMap_url) => {
            MDMU.debugLog("New envMap");

            let object  = el.getObject3D('mesh');

            if ( object !== undefined ) {
                let textureLoader           = new THREE.TextureLoader();
                let textureEquirec          = textureLoader.load( envMap_url );
                textureEquirec.mapping      = THREE.EquirectangularReflectionMapping;
                textureEquirec.magFilter    = THREE.LinearFilter;
                textureEquirec.minFilter    = THREE.LinearMipMapLinearFilter;

                object.traverse( function ( node ) {
                    if ( node.isMesh ) { 
                        if (node.material) {
                            node.material.envMap = textureEquirec;
                            node.material.needsUpdate = true;
                        }
                    }
                });
            }
            else {
                console.log("Entity 3Dmesh doesn't exist yet");
            }
        };

        //model change
        if ( (oldData.src !== data.src) && data.src !== null ) {

            MDMU.debugLog("New Model");

            let loader  = new THREE.GLTFLoader();
            // let src     = data.src;
            // let envMap  = data.envMap; 
            // let color   = data.color;

            loader.load( 
                data.src.getAttribute('src'), 
                function(gltf) {
                    let object                  = gltf.scene;
                    let textureLoader           = null;
                    let textureEquirec          = null;
                    if ( data.envMap !== null ) {
                        textureLoader               = new THREE.TextureLoader();
                        textureEquirec              = textureLoader.load( data.envMap.getAttribute('src') );
                        textureEquirec.mapping      = THREE.EquirectangularReflectionMapping;
                        textureEquirec.magFilter    = THREE.LinearFilter;
                        textureEquirec.minFilter    = THREE.LinearMipMapLinearFilter;
                    }

                    object.traverse( function ( node ) {
                        if ( node.isMesh ) { 
                            if (node.material) {
                                node.material.color = new THREE.Color(data.color);
                                if ( data.envMap !== null ) {
                                    node.material.envMap = textureEquirec;
                                }
                                node.material.needsUpdate = true;
                            }
                        }
                    });

                    el.setObject3D('mesh', object);
                    OContext.loaded = true;
                },
                // called when loading is in progresses
                function ( xhr ) {
                    //console.log( ( xhr.loaded / xhr.total * 100 ) + '% loaded' );
                },
                // called when loading has errors
                function ( error ) {
                    console.log( error.message );
                });
        }  

        //envMap change
        if ((oldData.envMap !== data.envMap) && (data.envMap !== null)) {
            if (OContext.loaded === true) {
                changeEnvMap(data.envMap.getAttribute('src'));
            }
            else {
                //not loaded try later
                let loaderCheckFunc_EnvMap = setInterval( () => {
                    if (OContext.loaded === true) {
                        //now load
                        changeEnvMap(data.envMap.getAttribute('src'));
                        clearInterval(loaderCheckFunc_EnvMap);
                        loaderCheckFunc_EnvMap = null;
                    }
                }, 1000);
            }
        }

        //color change
        if (oldData.color !== data.color) {
            if (OContext.loaded === true) {
                changeColor( data.color );
            }
            else {
                //not loaded yet - try later
                let loaderCheckFunc_color = setInterval( () => {
                    if (OContext.loaded === true) {
                        //now load
                        changeColor( data.color );
                        clearInterval(loaderCheckFunc_color);
                        loaderCheckFunc_color = null;
                    }
                }, 1000);
            }
        }
    },
    // tick: function(time, timeDelta) {},
    // tock: function(time, timeDelta) {},
    remove: function() {
        this.el.removeObject3D('mesh');
    },
    // pause: function() {},
    // play: function() {},
    // updateScheme: function(data) {}
});

[donmccurdy]

The component doesn't really need to deal with loading models, it can just listen for object3dset events and add envMap. Example here: https://github.com/donmccurdy/aframe-extras/blob/99535cd878eb36cb25cdcda4a32a40eb248c990a/src/misc/cube-env-map.js

[PlumCantaloupe]

Thanks @donmccurdy. Is this the preferred pathway for extending functionality of any component in A-Frame? (via events and adding more components?)

[donmccurdy]

It depends, but definitely there are lots of components that modify a model after it's loaded. If you use object3dset events for that, then you don't have to worry about what format the model was in.

Maybe A-Frame should have an env-map, maybe not, I'm not sure on that one. But in any case there will be many situations where you need a small component to modify a three.js object like that. :)

[PlumCantaloupe]

Ah I see. This is some great info. to chew on. Thanks again.