GLTF for standalone material files

[JoshKlint]

I'm just starting to look into the GLTF spec, and so far I approve of most of the design decisions. One thing I am not seeing is whether the format supports / is meant to support standalone material files. There are many situations when a material should be shared by multiple model files or just kept as a standalone asset the user can load programmatically. Considering how bad the state of cross-application materials is in the game industry, this seems like a golden opportunity. I was disappointed to see that Substance Designer does not export materials in GLTF format. It seems that the spec designers think of a material as being a bunch of settings attached to a model.

If it's conceivable, I would like to replace our model and material files entirely with GLTF.

What can you tell me about this?

[samsonsite1]

Hello Josh,

I'm also new to glTF, and I've been looking it over for a couple weeks now.

You could probably do this with extensions. You could dump "materials" and everything below it (textures,samplers,images) into its own material file. Then, add a uri reference to this material file using your own extension.

If the importing app didn't support your extension, then it would just ignore it, and use whatever default material is provided in the scene.

But, to make it useful, other apps would have to support it, like Substance Desiger. Or you would have to write your own exporters.

// scene file

{
  "meshes": [
    {
      "name": "mesh0"

      "primitives": [
        {
          "attributes": {
            "NORMAL": 1,
            "POSITION": 2,
            "TEXCOORD_0": 3
          },

          "indices": 0,
          "mode": 4,
          "material": 0,

          "extensions": {
            "MY_Extension": {
               "material": {
                   "uri": "matfile.mat"
               }
            }
          }
        }
      ],
    }
  ],

  "extensionsRequired": [
    "MY_Extension"
  ],
  "extensionsUsed": [
    "MY_Extension"
  ]
}

// matfile.mat

{
    "materials": [
        {
            "pbrMetallicRoughness": {
                "baseColorTexture": {
                    "index": 0
                },
                "metallicFactor": 0.0
            },
            "name": "Texture"
        }
    ],
    "textures": [
        {
            "sampler": 0,
            "source": 0
        }
    ],
    "images": [
        {
            "uri": "image.png"
        }
    ],
    "samplers": [
        {
            "magFilter": 9729,
            "minFilter": 9986,
            "wrapS": 10497,
            "wrapT": 10497
        }
    ]
}

Just an idea off the top of my head. Don't know if it'll work or not for you.

[donmccurdy]

No need for an extension — the spec allows this already:

When scene is undefined, runtime is not required to render anything at load time.

Implementation Note: This allows applications to use glTF assets as libraries of individual entities such as materials or meshes.

For example, a glTF file might contain only a materials array:

{
  "asset": {...},
  "materials": [...]
}

However, I'm not aware of exporters that currently create files this way. In three.js we're working on loading arbitrary pieces of a glTF file: https://github.com/mrdoob/three.js/pull/14492

[JoshKlint]

It's good that this is at least accounted for. Is there any official file extension for GLTF materials, like "GLMF" or something?

[emackey]

+1 for Don's comments, but I'll toss in a caveat here. Sometimes in Substance Painter I'll have a clean surface that's been splattered by mud a certain way, or a metallic surface with rust spots, etc. My point is that "materials" in the physical, non-GPU sense of the word, can be considered as per-texel in glTF 2. You often can have a metallic texel next to a rust texel, with the rust pattern highly specific to the current model geometry.

The things that glTF labels as "materials" are more akin to render states or texture sets for a whole region of a model comprised of multiple physical materials. In core 2.0 without extensions, each one is a copy of the core PBR material with some blend/cull options toggled and a specific stack of textures. Generally I don't expect this type of glTF material to be reusable across models, certainly not the way SP material presets are easily reusable.

[samsonsite1]

Just a question, assuming a glTF scene has been split up into parts, how does an importing app recognize multiple asset files? Should it scan the entire local directory for assets? And how does it know those asset files belong to each other, if the main file doesn't contain any uri references to them?

Or are you using a file naming scheme, like myscene0.gltf, myscene1.gtlf, myscene2.gltf, etc...

[donmccurdy]

The .gltf file will contain references to all textures and .bin files it uses, and the importing app can load them as needed.

[JoshKlint]

In Leadwerks Game Engine, a material consists of the following:

• Array of textures, which corresponds to GLSL image units.
• Some appearance settings like depth mask, double-sided, color, lighting model, etc.
• Shader(s) to use (engine auto-selects one if this is not present) for regular rendering, shadow pass, etc.

So that's where I'm coming from.

My dream is to be able to select a GLMF file in Windows Explorer and have stock Windows display a sphere with the material on it in the preview pane.

[emackey]

In core glTF 2.0, the material definition is similar. It consists of:

• Array of textures for use with the core metal/rough shader.
• Some appearance settings like alpha mode, double-sided, color factor/multipliers, etc.
• No choice of shader in core, always use metal/rough. Certain extensions offer alternate shaders/paradigms.

The Damaged Helmet model has an example of what I'm talking about. This model has metal, glass, lighted elements, and rubber hoses, among other details. How many glTF materials are being used to convey all of these different substances?

Just one.

This model was originally textured in Substance Painter, and several different SP material presets were clearly used in its construction. But the final result was exported to a single base color, a single metal/rough, a single occlusion, a single normal map, and a single emissive map, that collectively form a single glTF 2.0 core material.

I do like the idea of using the glTF container to share libraries of things, but in the case of materials, I think there are some practical considerations that may complicate the process.

[JoshKlint]

98% of materials are going to use the default PBR shader paradigm, so I am fine with that. Custom shaders are an exception and should not dictate the capabilities of the entire system. One of the strengths of GLTF is that Khronos is actually making firm decisions this time around.

You could make a good case for a Blinn-Phong mode, but anything in addition to that is going to depend on the individual engine the material is meant to be used in.

[emackey]

Yes. I can imagine a glTF with a bunch of KHR_technique_webgl shaders in glTF materials, with an asset block but no meshes or nodes. That should already validate as proper glTF in the Khronos glTF validator.

[JoshKlint]

Okay, I tried isolating a material from the damaged helment GLTF file and this is what I came up with: What do you think?

{
    "asset": {
        "version": "2.0"
    },
    "images": [
        {
            "uri": "Assets/Models/PBR/damagedHelmet/textures/Default_albedo.jpg"
        },
        {
            "uri": "Assets/Models/PBR/damagedHelmet/textures/Default_MetalSmooth_converted_metalRoughness.jpg"
        },
        {
            "uri": "Assets/Models/PBR/damagedHelmet/textures/Default_normal.jpg"
        },
        {
            "uri": "Assets/Models/PBR/damagedHelmet/textures/Default_emissive.jpg"
        },
        {
            "uri": "Assets/Models/PBR/damagedHelmet/textures/Default_AO.jpg"
        }
    ],
    "samplers": [
        {
            "magFilter": 9729,
            "minFilter": 9985,
            "wrapS": 10497,
            "wrapT": 10497
        }
    ],
    "textures": [
        {
            "sampler": 0,
            "source": 0
        },
        {
            "sampler": 0,
            "source": 1
        },
        {
            "sampler": 0,
            "source": 2
        },
        {
            "sampler": 0,
            "source": 3
        },
        {
            "sampler": 0,
            "source": 4
        }
    ]
    "pbrMetallicRoughness": {
        "baseColorTexture" : {
            "index" : 0,
            "texCoord" : 0
        },
        "baseColorFactor": [1, 1, 1, 1],
        "metallicRoughnessTexture" : {
            "index" : 1,
            "texCoord" : 0
        },
        "metallicFactor": 1,
        "roughnessFactor": 1
    },
    "normalTexture" : {
        "index" : 2,
        "texCoord" : 0,
        "scale" : 0.8
    },
    "emissiveTexture" : {
        "index" : 3,
        "texCoord" : 0
    },
    "emissiveFactor": [1, 1, 1],
    "occlusionTexture" : {
        "index" : 4,
        "texCoord" : 0,
        "strength" : 0.632
    },
    "doubleSided": false,
    "name": "Helmet_mat"
}

[JoshKlint]

If we modify the damaged_helment GLTF file to use external material files, then it looks like this:

{
    "asset": {
        "version": "2.0"
    },
    "materials": [
        {
            "uri": "Helmet_mat.glmf"
        }
    ],
    "accessors": [
        {
            "bufferView": 2,
            "byteOffset": 0,
            "componentType": 5126,
            "count": 13600,
            "max": [ 0.944977, 0.900995, 1 ],
            "min": [ -0.944977, -0.900974, -1 ],
            "type": "VEC3"
        },
        {
            "bufferView": 2,
            "byteOffset": 163200,
            "componentType": 5126,
            "count": 13600,
            "max": [ 1, 1, 1 ],
            "min": [ -1, -1, -1 ],
            "type": "VEC3"
        },
        {
            "bufferView": 1,
            "byteOffset": 0,
            "componentType": 5126,
            "count": 13600,
            "max": [ 0.999976, 0.998666 ],
            "min": [ 0.00244864, 0.00055312 ],
            "type": "VEC2"
        },
        {
            "bufferView": 3,
            "byteOffset": 0,
            "componentType": 5126,
            "count": 13600,
            "max": [ 1, 1, 1, 1 ],
            "min": [ -1, -1, -1, -1 ],
            "type": "VEC4"
        },
        {
            "bufferView": 0,
            "byteOffset": 0,
            "componentType": 5123,
            "count": 46356,
            "max": [ 13599 ],
            "min": [ 0 ],
            "type": "SCALAR"
        }
    ],
    "buffers": [
        {
            "byteLength": 745512,
            "uri": "damagedHelmet.bin"
        }
    ],
    "bufferViews": [
        {
            "buffer": 0,
            "byteLength": 92712,
            "target": 34963,
            "byteOffset": 652800
        },
        {
            "buffer": 0,
            "byteLength": 108800,
            "target": 34962,
            "byteStride": 8,
            "byteOffset": 0
        },
        {
            "buffer": 0,
            "byteLength": 326400,
            "target": 34962,
            "byteStride": 12,
            "byteOffset": 108800
        },
        {
            "buffer": 0,
            "byteLength": 217600,
            "target": 34962,
            "byteStride": 16,
            "byteOffset": 435200
        }
    ],
    "meshes": [
        {
            "name": "mesh_helmet_LP_14908damagedHelmet",
            "primitives": [
                {
                    "attributes": {
                        "POSITION": 0,
                        "NORMAL": 1,
                        "TEXCOORD_0": 2,
                        "TANGENT": 3
                    },
                    "indices": 4,
                    "material": 0,
                    "mode": 4
                }
            ]
        }
    ],
    "nodes": [
        {
            "name": "root",
            "children": [
                1
            ]
        },
        {
            "name": "node_damagedHelmet_-6498",
            "mesh": 0,
            "matrix": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
        }
    ],
    "scenes": [
        {
            "name":"defaultScene",
            "nodes": [
                0
            ]
        }
    ],
    "scene": 0,
}

[donmccurdy]

I think this arguably fits into https://github.com/KhronosGroup/glTF/issues/1518, i.e. rather than designing a mechanism specifically for materials to be referenced in an external file, perhaps (or perhaps not) there should be a more general mechanism for referencing a resource that handles external references?

On the other hand, more feedback would probably also help to prioritize this; I'm not sure how common it is for the material JSON to be large enough to benefit from using external files, or if this is more of an asset management convenience.

[JoshKlint]

I've been working with GLTF for a while now and have a better understanding of the format. It seems like we can use this for material files, adding some extensions where needed. I am interested to if Khronos gives some more direction on this sort of usage. The main reason I am doing this is because our editor has CSG level design features so the developer is using a lot of standalone material files not embedded in a model.

[emackey]

I think, if you're not dissuaded by my comments above, the next step would be to put together a sample of what a stand-alone material file would look like in glTF. At a minimum it would have the asset tag with the 2.0 version number, and beyond that it would be lacking scene, nodes, etc, and would just have materials with maybe textures, samplers, and images.

[JoshKlint]

Okay, below is an example of a valid standalone GLTF material.

Implementation Note: This allows applications to use glTF assets as libraries of individual entities such as materials or meshes.

Because the spec says a GLTF file can contain "libraries" (plural) of assets it makes sense to indicate which material is considered the "main" asset, so I added a "material" parameter similar to the "scene" parameter, indicating which material to load. This also makes implementation easier because if this value is present you know it is a GLTF material file.

After thinking about it a while, I think it's probably best to continue using the gltf / glb file extension for all asset types, and not getting tricky with new extensions.

{
    "asset": {
        "version": "2.0",
    },
    "material": 0,
    "images": [
        {
            "uri": "base.jpg"
        }
    ],
    "samplers": [
        {
            "magFilter": 9729,
            "minFilter": 9985,
            "wrapS": 10497,
            "wrapT": 10497
        }
    ],
    "textures": [
        {
            "sampler": 0,
            "source": 0
        }
    ],
    "materials": [
        {
            "pbrMetallicRoughness": {
                "baseColorTexture" : {
                    "index" : 0,
                    "texCoord" : 0
                },
                "baseColorFactor": [1, 1, 1, 1],
                "metallicFactor": 1,
                "roughnessFactor": 1
            },
            "doubleSided": false,
            "name": "MyMaterial"
        }
    ]
}

Loading this material from another GLTF file is a no-brainer:

    "materials": [
        {
            "uri": "Assets/MyMaterial.gltf"
        }
    ]

So this only needs two small changes to work:

• "material" parameter to indicate which material to return, and that this is a GLTF material file.
• Use of "uri" tag in the materials block.

[emackey]

Rather than adding type: "MATERIAL" and material: 0, you should use the normal glTF extension mechanism. You would create a root-level extension indicating a material library, and the body of that extension could reference the default material. That way, your file will pass glTF validation.

That said, I'm most curious about the contents of base.jpg. It's been my experience that glTF material textures are tightly bound to the UV atlas chosen for the geometry they represent. What makes base.jpg reusable by other models with other geometry and UV maps? Maybe it's just a repeating tileable texture that can support an arbitrary UV map? The models that I typically use have a dedicated map. For example, there are no repeating textures in the DamagedHelmet sample model. That model has a single Damaged Helmet Material that places rubber on the hoses and glass on the visor, metal on the sides, etc. There's no way to re-use the Damaged Helmet Material on any object that isn't exactly the Damaged Helmet with the Damaged Helmet UV atlas.

So I question the practicality of this, but, from a technical perspective, getting it through the glTF Validator (possibly with the help of an extension) seems easy enough.

[JoshKlint]

I removed the "type" value when I realized it was not needed.

Standalone materials are used for any tiling textures in level design: It would be nice if there was a standard material format, even if it doesn't support every feature under the sun.

[emackey]

Sounds good, that's a good use for it.