Supported glTF extensions

[Ybalrid]

I am creating this issue to track the support of officially published extensions, and their support by glTF-insight.

One of the most interesting parts of the glTF format is that it can be extended to add missing features or applications specific metadata.

For reminder, there's 3 majors level of extensions: KHR ones are ratified by Khronos. We should expect theses to be present in assets, are they both contains features that are really missing from the base format, and that are generally supported by implementations.

EXT extensions are extensions ratified by multiple vendors. They could be promoted to the KHR level, but aren't yet. They are "industry consensus" addition to the file format. I'm personally really interested by the one about IBL as it permits a glTF scene to provide it's own HDR environment map to define more natural lighting.

Other are vendor specific, and thus have been published because they could be interesting for the comunity, but they aren't that important for us as of now.

glTF-insight extension support

Below are the most relevant extensions our software should support:

general:

• [ ] KHR_draco_mesh_compression Note: This is supported directly inside tinygltf

material:

• [x] KHR_materials_unlit
• [ ] KHR_materials_pbrSpecularGlossiness (TODO only suport fallback to metal_roughness ATM. Dedicated shader needed)
• [ ] KHR_texture_transform

lighting:

• [ ] EXT_lights_image_based

Vendor specific extension with planned support:

• [ ] MSFT_audio_emitter (see issue #42)
• [ ] MESHOPT_quantized_geometry
• [ ] MESHOPT_compression

[cesss]

Hi! (I'm the one who opened this issue at tinygltf and from there you told me about this project, which is really exciting, and it's progressing at a really promising pace!! Congratulations, and thanks a lot!!

Regarding extensions, I must say I'm a bit disappointed about glTF 2.0 because there are two features that I need for my global illumination scenes, and they are not supported yet (I already posted about this at the issues in the official glTF repository):

1. Quads: If you do either subdiv surfaces or radiosity patches, you want quads. AFAIK, there's no extension for quads. Are you aware of any workaround for quads? How are you supposed to save a Catmull-Clark subdivision surface without quads?
2. PBR transparency and refraction: I feel disappointed that the current glTF PBR materials don't support PBR transparency/translucency/refraction. Yes, it's a complex topic, not suitable for realtime in GPU yet, but... come on!!!, there are plenty of BRDF/BTDF/BSDF models published throughout the years, and if they chose a BRDF similar to the Disney one as the BRDF for PBR in glTF, they could also have chosen to support some BTDF model as well. When posting about this in the glTF repository, I was suggested to take a look at the ADOBE_materials_thin_transparency extension. But, as you mention, it's a vendor-only extension. Too bad!!

These two things (quads and PBR transparency) are must-have features for my global illumination scenes, so somehow I believe glTF is not yet ready for proper GI work. Until they add these features, maybe it would be possible to create a "container format" that combines glTF with other files, like Wavefront OBJ, enclosing everything in a TGZ for example (similar to the Microsoft Office docx/xlsx formats which are actually ZIP containers with several files inside). I mean, maybe it would be possible to save a glTF file together with a companion file that adds all the missing things, and enclosing both files in a container.

[syoyo]

Quads

You can use this.

https://github.com/KhronosGroup/glTF/pull/1620

Although ngon extension is currently neither supported in TinyGLTF nor gltf-insight. You can implement it and send us PR.

PBR transparency and refraction:

This is rather off topic, but we have implemented exporting Maya Arnold PBR(close to Disney principled BSDF) parameter to glTF as an extension.

https://github.com/lighttransport/glTF-Maya-Exporter/blob/development/src/glTFExporter/glTFExporter.cpp#L1756

You can use this definition and follow https://github.com/Autodesk/standard-surface to implement your own PBR model.

FYI, we've successfully implemented corresponded Arnold StandardSurface PBR material in our private monte carlo raytracer.

[cesss]

Quads

You can use this.

KhronosGroup/glTF#1620

Although ngon extension is currently neither supported in TinyGLTF nor gltf-insight. You can implement it and send us PR.

PBR transparency and refraction:

This is rather off topic, but we have implemented exporting Maya Arnold PBR(close to Disney principled BSDF) parameter to glTF as an extension.

https://github.com/lighttransport/glTF-Maya-Exporter/blob/development/src/glTFExporter/glTFExporter.cpp#L1756

You can use this definition and follow https://github.com/Autodesk/standard-surface to implement your own PBR model.

FYI, we've successfully implemented corresponded Arnold StandardSurface PBR material in our private monte carlo raytracer.

Both suggestions are great!! Regarding the ngon encoding extension, didn't know about that, but I was also thinking about the possibility of encoding quads by simply considering that every two triangles in a mesh constitute a quad. That extension actually makes this more powerful, extending it to ngons.

I think I'm going to try to use this extension. I'm not sure if a PR is required for it to be supported in TinyGLTF, as it's only a matter of interpreting the order of vertices, so maybe it's not even a task for TinyGLTF, but for the software that uses TinyGLTF. I'll study it.

Regarding your PBR extension, have you considered proposing it at the glTF official repository as an extension? They are in fact asking in a couple of issues about wishlists for future >2.0 glTF versions.

[Ybalrid]

@cesss The way extensions are defined in a glTF assets, and the fact that tinygltf just expose you the glTF data in a low level (but convenient) way in C++ makes it that, unless the extensions has to do with decoding the binary buffers (e.g:. draco compression), or decoding images (e.g.: KTX texture format), they mostly are about adding more metadata to some part of the JSON description of the object. In that case, the metadata is added to the "extension" parameters where they belong (like, on a mesh or a material or a texture object).

In that case, it's mostly the responsibility of the client program using TinyGLTF to support the extension.

We are thinking about, only for the case of the KHR extensions, to try to detect them and explicitly expose them through the API, but it's a convenience feature discussed here https://github.com/syoyo/tinygltf/issues/152

[cesss]

I really need quads for my work. If I try to implement the ngon-extension myself in gltf-insight, what would be your preferred way for doing it? I mean: should I add quad faces (or even n-gon faces although I don't need them) in the data structures? Or would you prefer that I store n-gons as pointers to triangles?

I'm asking because I really don't know how quads/ngons would interfere with your skinning/morphing implementation at this point, and I'd like to follow the path you prefer.

[cesss]

I just had a very simple idea to implement quads and n-gons in a very simple way: no need to new data structures, nor even to store the "collection of triangles" which make each n-gon. Instead of doing that, I think it would be enough just to flag every mesh as "normal" or "with n-gon extension enabled". When a mesh is flagged as n-gons, you can work on it the same way as if it was triangles (because IT IS triangles indeed!!), but however, when rendering in OpenGL, you can use the vertex order to figure out how many vertices each face has, and render it accordingly (quite likely you would only notice the difference if you render in wireframe mode).

Also, when saving a OBJ or a glTF, you can check if the flag is enabled, and save the mesh accordingly (in the case of OBJ, save real quads, while in the case of glTF just output the extension name and keep the vertices order).

I'm of course supposing that tinygltf doesn't reorder vertices when loading, does it?

What do you think about this idea? I can try to implement it and issue a PR if you like it.

[Ybalrid]

As a side note : I am not familiar with the ngon glTF extension, but I have some input for you:

For skinning and morphing, it's done vertex per vertex. Thus, skinning and morphing don't depend on the fact that 3 of these vertex put together draw a triangle or not.

For rendering tho, as far as I know, to render a quad in OpenGL, you end up drawing triangles anyway, same for any kind of polygon with more than 3 vertices. There are a few drawing mode in OpenGL that speed up drawing of a bunch of triangle by permitting to specify the next triangle with only one vertex after the 3 first ones

Notably,

GL_TRIANGLE_FAN to draw something like this

This can be drawn as a triangle list using the index order 0; 1; 2; 0; 2: 3; 0; 3; 4... or a a triangle fan using the index order `0; 1; 2; 3; 4..."

GL_TRIANGLE_STRIP to draw something like that:

Again, as a triangle list, this needs to have the index buffer 1; 2; 3; 3; 2; 4 (for example, there are other valid solutions), but as a triangle strip this can be specified just by using 1; 2; 3; 4. This is the best

You can draw both of theses as a list of triangle too. The only thing that changes is the index buffer.

Note that flat normal generation code in glTF-insight (we generate normal vectors when they aren't specified) assume the index buffer exposes a triangle list. This technically already need to be extended as glTF can already use the triangle list/fan modes. If that scenario arise, current code will print a warning in the console if that's not the case. It seems to be pretty rare than an exporter will generate a mesh primitive that don't use a triangle list, and will also not export explicit vertex normal.

I'm of course supposing that tinygltf doesn't reorder vertices when loading, does it?

It doesn't even load "vertices". It only exposes the data defined in the glTF asset for you in a low level manner. It's not a high level "mesh loader" like assimp. So no, it will not reorder them for you.

[Ybalrid]

At some point in the near future, it would be nice to implement #40

[Ybalrid]

Added a list of "planned support" for a few vendor-level extensions at the end of original post. The ones mentioned in other issues.