GLTF with material that uses `transmission` misses color in model-viewer but not in other renderers like babylon.js

[Bersaelor]

Description

When I preview our model using modelviewer the Plastic.002 material is missing it's color. Other renderers like bablyonjs look fine.

I have been trying the new transmission property, so somehow the different parameters between Plastic.001 (which just has alpha < 1) and the new way of achieving transparency using the transmission parameter remove the color from the material.

Live Demo

Drag the model into the above mentioned viewers and it will look like this:

Version

• model-viewer: v2.1.1

Browser Affected

• [x] Chrome, version: 108.0.5359.124
• [ ] Edge
• [ ] Firefox
• [ ] IE
• [x] Safari

OS

• [ ] Android

• [ ] iOS

• [ ] Linux

• [x] MacOS

• [ ] Windows

[elalish]

Hmm, the difference seems to be:

      "alphaMode": "OPAQUE",
      "doubleSided": true,
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 1
        }
      },
      "name": "Plastic.002",

vs

      "alphaMode": "BLEND",
      "doubleSided": true,
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 0.5
        }
      },
      "name": "Plastic.001",

I'm not quite sure how one should interpret "transmissionFactor": 1 together with "alphaMode": "OPAQUE". But even when I set it to BLEND, it still looks wrong (no color); might help to reduce the transmissionFactor. In any case, this is a three.js rendering issue, so I'd recommend you file an issue there are get a discussion going.

[Bersaelor]

Thank you for your reply @elalish!

So the last weeks I've been using transmission: 1 with three.js version "^0.146.0" in our plain Three.js app with no problem. There I setup the THREE.MeshPhysicalMaterial manually , e.g.

    var material: THREE.MeshPhysicalMaterial = new THREE.MeshPhysicalMaterial({
        opacity: opacity,
        color: Number(hex), 
        reflectivity: m.parameters.reflectivity,
        metalness: m.parameters.metallicity,
        roughness: m.parameters.roughness,
        transmission: m.parameters.transmission
    });

and never had any issues with the colors.

The materials can be fine-tuned with the live-preview in Three.js. Then I use blender and a python script to assemble the whole scene in blender, based on the chosen materials and 3D objects, which are then exported as *.glb.

The material that made the above images looked fine in Three.js before:

PS: And you're right, I tried both BLEND and OPAQUE to see if it makes any difference in the gltf but that doesn't seem to be it. Also, either way the color should be applied correctly.

[elalish]

Thanks, that's helpful. We use three.js under the hood, so what this really implies is that the exporters and importers are not managing to do a lossless round-trip of a three.js material. Not sure if the bug is on the three.js side, the Blender side, or if you've just selected some combination that's not representable in glTF. You can set a break point in MV, maybe down in the three.js render method and take a look at the MeshPhysicalMaterial you actually ended up with. Hopefully that will help with filing bugs on the upstream tools.

[Bersaelor]

. You can set a break point in MV, maybe down in the three.js render method and take a look at the MeshPhysicalMaterial you actually ended up with.

Good idea, I'm not super deep in the tools necessary to put breakpoints into dependencies, but will see what I can do (I.e. the three.js thats in the node_modules folder is minified and not running the src folder, I have to google how to put breakpoints in the un-minified code).

In the mean time, I built the following minimal example:

// npm installed is "three": "^0.146.0",
import { Collada, ColladaLoader } from 'three/examples/jsm/loaders/ColladaLoader';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';

import planga from './planga.glb';

//...
                scene.current = new THREE.Scene();
                scene.current.background = new THREE.Color( 0xffffff );

                const loader = new GLTFLoader( );
                const dracoLoader = new DRACOLoader();
                dracoLoader.setDecoderPath( 'https://www.gstatic.com/draco/versioned/decoders/1.4.1/' );
                loader.setDRACOLoader( dracoLoader );

                loader.load(planga, function (main) {
                    const g = main.scene.childNamed("Glasses")
                    if (scene.current) scene.current.add(g);
                })
// ... plus the usual camera positioning and render mounting

planga is the same model linked above and the result looks fine (you can see the transmission since the internal metal parts of correctly blurred on the left and right end of the lenses:

so it seems plain THREE.js can load the linked model fine.

[Bersaelor]

Alright, I managed to put a breakpoint in node_modules/@google/model-viewer/src/three-components/Renderer.ts/Renderer/render method and log the scene:

scene._model.children[0].children[0].children[0] should be the part named FRONT. console.log(scene._model.children[0].children[0].children[0].material):

``` MeshPhysicalMaterial {isMaterial: true, uuid: 'd8b0f0d6-527e-4f38-b7f1-2c7504fc724f', name: 'Plastic.002', type: 'MeshPhysicalMaterial', blending: 1, …} _alphaTest: 0 _clearcoat: 0 _iridescence: 0 _listeners: {dispose: Array(1)} _sheen: 0 _transmission: 1 alphaMap: null alphaTest: ƒ () {\n return this._alphaTest;\n } alphaToCoverage: false aoMap: null aoMapIntensity: 1 attenuationColor: Color {isColor: true, r: 1, g: 1, b: 1} attenuationDistance: Infinity blendDst: 205 blendDstAlpha: null blendEquation: 100 blendEquationAlpha: null blending: 1 blendSrc: 204 blendSrcAlpha: null bumpMap: null bumpScale: 1 clearcoat: ƒ () {\n return this._clearcoat;\n } clearcoatMap: null clearcoatNormalMap: null clearcoatNormalScale: Vector2 {x: 1, y: -1} clearcoatRoughness: 0 clearcoatRoughnessMap: null clipIntersection: false clippingPlanes: null clipShadows: false color: Color {isColor: true, r: 0.1119324266910553, g: 0.061246052384376526, b: 0.6307571530342102} colorWrite: true defines: {STANDARD: '', PHYSICAL: ''} depthFunc: 3 depthTest: true depthWrite: true displacementBias: 0 displacementMap: null displacementScale: 1 dithering: false emissive: Color {isColor: true, r: 0, g: 0, b: 0} emissiveIntensity: 1 emissiveMap: null envMap: null envMapIntensity: 1 flatShading: false fog: true ior: 1.5 iridescence: ƒ () {\n return this._iridescence;\n } iridescenceIOR: 1.3 iridescenceMap: null iridescenceThicknessMap: null iridescenceThicknessRange: (2) [100, 400] isMaterial: true isMeshPhysicalMaterial: true isMeshStandardMaterial: true lightMap: null lightMapIntensity: 1 map: null metalness: 0 metalnessMap: null name: 'Plastic.002' normalMap: Texture {isTexture: true, uuid: 'bc0eec5b-3209-48e1-a84e-c5ae967fc084', name: 'plastic-ozean-normal', source: Source, mipmaps: Array(0), …} normalMapType: 0 normalScale: Vector2 {x: 1, y: -1} opacity: 1 polygonOffset: false polygonOffsetFactor: 0 polygonOffsetUnits: 0 precision: null premultipliedAlpha: false reflectivity: ƒ () {\n return clamp(2.5 * (this.ior - 1) / (this.ior + 1), 0, 1);\n } roughness: 0.4300000071525574 roughnessMap: null shadowSide: 0 sheen: ƒ () {\n return this._sheen;\n } sheenColor: Color {isColor: true, r: 0, g: 0, b: 0} sheenColorMap: null sheenRoughness: 1 sheenRoughnessMap: null side: 2 specularColor: Color {isColor: true, r: 1, g: 1, b: 1} specularColorMap: null specularIntensity: 1 specularIntensityMap: null stencilFail: 7680 stencilFunc: 519 stencilFuncMask: 255 stencilRef: 0 stencilWrite: false stencilWriteMask: 255 stencilZFail: 7680 stencilZPass: 7680 thickness: 0 thicknessMap: null toneMapped: true transmission: ƒ () {\n return this._transmission;\n } transmissionMap: null transparent: false type: 'MeshPhysicalMaterial' userData: {associations: {…}} uuid: 'd8b0f0d6-527e-4f38-b7f1-2c7504fc724f' version: 1 vertexColors: false visible: true wireframe: false wireframeLinecap: 'round' wireframeLinejoin: 'round' wireframeLinewidth: 1 id: 67 ```

The line

color: Color {isColor: true, r: 0.1119324266910553, g: 0.061246052384376526, b: 0.6307571530342102}

looks like it should be fine to me, still no color in model-viewer.

In comparison let's also log the MeshPhysicalMaterial from the minimal Three.js example above, that just loads the glb and shows it fine (JSON.stringified this time):

``` { "uuid": "726f8e51-40a3-4c9b-903b-ec425b2c4ff7", "type": "MeshPhysicalMaterial", "name": "Plastic.002", "color": 1839008, "roughness": 0.4300000071525574, "metalness": 0, "sheen": 0, "sheenColor": 0, "sheenRoughness": 1, "emissive": 0, "specularIntensity": 1, "specularColor": 16777215, "clearcoat": 0, "clearcoatRoughness": 0, "iridescence": 0, "iridescenceIOR": 1.3, "iridescenceThicknessRange": [ 100, 400 ], "normalMap": "f7102614-c76b-4efa-9a3d-922399780694", "normalMapType": 0, "normalScale": [ 1, -1 ], "envMapIntensity": 1, "reflectivity": 0.5, "transmission": 1, "thickness": 0, "attenuationColor": 16777215, "side": 2, "depthFunc": 3, "depthTest": true, "depthWrite": true, "colorWrite": true, "stencilWrite": false, "stencilWriteMask": 255, "stencilFunc": 519, "stencilRef": 0, "stencilFuncMask": 255, "stencilFail": 7680, "stencilZFail": 7680, "stencilZPass": 7680, "textures": [ { "uuid": "f7102614-c76b-4efa-9a3d-922399780694", "name": "plastic-ozean-normal", "image": "ae59df60-74e5-454e-87d5-0faa853eafa7", "mapping": 300, "repeat": [ 1, 1 ], "offset": [ 0, 0 ], "center": [ 0, 0 ], "rotation": 0, "wrap": [ 1000, 1000 ], "format": 1023, "type": 1009, "encoding": 3000, "minFilter": 1008, "magFilter": 1006, "anisotropy": 1, "flipY": false, "premultiplyAlpha": false, "unpackAlignment": 4, "userData": { "mimeType": "image/jpeg" } } ], "images": [ ] } ```

[elalish]

Yeah, this is strange, I too looked at the materials between the working and non-working version and couldn't find any difference. Can you repro with a simpler model? Perhaps with just one material?

[mrdoob]

Where can I find a link to the working version of the model?

[Bersaelor]

Where can I find a link to the working version of the model?

It's the same model one time I imported it via a plain THREE.js implementation (works, see https://github.com/google/model-viewer/issues/4047#issuecomment-1384066921 ) . But then importing via modelviewer the material failed to work (as in og post)

[elalish]

I just had an inspiration on why this is different in MV vs the three.js editor: MV renders transparently, compositing afterward into whatever DOM content is behind it, while three's editor renders a background into its own buffers. You can see the difference in our own editor by loading up your model and in the second tab, choosing a different environment image and then checking "use environment as skybox" - suddenly it looks like the other renderers.

This is a helpful feature of MV for web design, but I believe there's also improvement that could be made in three's shaders for transparent materials with no background images behind them. As a workaround, you can set a simple single-color skybox-image and use environment-image="neutral" or whatever you want.

[mrdoob]

So if there's no background we render the environment as background in the transmission buffer?

[elalish]

So if there's no background we render the environment as background in the transmission buffer?

I hope the answer to this is no. I just want to see if our little non-PBR hack for rendering to a transparent buffer can be improved to save the transmissive color better.

[mrdoob]

Hmm, I can't see how to make that work...

If it was me, I would make it so <model-viewer>'s background (white by default?) goes opaque and logs a warning in the console saying that models with refractive materials only work when <model-viewer> has a opaque background and display the API to change the background color.

And then remove the hack from three.js shader 😅

[elalish]

I think I have a promising solution: https://github.com/mrdoob/three.js/pull/25819

[elalish]

Fixed by #4314 - by pulling in the above change.