glTF Spec and MVR

[bbudzon]

Is your feature request related to a problem? Please describe. The GDTF spec, when adding support for glTF, requires that no extensions of glTF are used. While this works well for GDTF files, it may not work quite as well for MVR. This is because many programs that support glTF only support the SpecGloss or the MetalRough workflows. Not all programs support both MetalRough as well as SpecGloss.

Describe the solution you'd like While I agree that in general all extensions should be avoided when using glTF with MVR, I think a special exception needs to be made for the SpecGloss extension. Not only will this increase the interoperability of MVR with more programs, but it also an "official" extension created and published by the Khronos Group themselves. So, this is not some third party extension that programs are unlikely to support.

All of this being said, it is my opinion that MetalRough should always be provided; regardless of whether or not SpecGloss is present. This is because the glTF spec allows for MetalRough as a "fallback" in case the reading program does not support SpecGloss if it is present. So, it is of my opinion that MetalRough materials should be mandatory in glTF contained in an MVR with an optional SpecGloss material being provided in the event the program writing the glTF follows the SpecGloss workflow exclusively.

Describe alternatives you've considered When I first investigated MetalRough vs SpecGloss workflows, it was not clear whether or not this conversion could be done programmatically. It appeared as though "human intervention" was required. However, in my more recent searches, it appears some algorithms have surfaced which allow SpecGloss to be converted programmatically to MetalRough.

In theory, this means we could continue disallowing all extensions (including SpecGloss) in glTF contained in MVR and demand MetalRough while considering SpecGloss invalid. My concern here is only that some programs may truly only support SpecGloss at read time and so having the ability to include SpecGloss may still be valuable. Furthermore, we cannot be guaranteed that all programs that might want to write a glTF contained in MVR contain the necessary code for converting SpecGloss to MetalRough.

For these reasons, I believe the proper solution is making an exception in regards to glTF extensions for SpecGloss materials.

Additional context I think the most important thing to consider with glTF contained inside of MVR is interoperability. For this reason, almost all extensions should not be allowed. We should follow the base spec and the base spec only. I feel that the SpecGloss extension is a widely supported one, and one that some programs mandate at read time in order for materials to be read. So, an exception for this extension seems reasonable; especially given the fact that when SpecGloss is provided MetalRough can also be provided as a fallback.

[LxLasso]

The KHR_materials_pbrSpecularGlossiness extension has been archived and use thereof is discouraged. See https://github.com/KhronosGroup/glTF/tree/main/extensions.

[bbudzon]

@LxLasso, correct. The extension you mention has been superseded by the KHR_materials_specular extension, which supports the SpecGloss workflow I mentioned above.

EDIT: I should be more clear. Converting SpecGloss to MetalRough is lossy. If, however, the KHR_materials_specular extension is used, then the conversion from SpecGloss to MetalRough is lossless (as far as I can tell).

[LxLasso]

Actually, KHR_materials_specular is only a subset of KHR_materials_pbrSpecularGlossiness. In KHR_materials_pbrSpecularGlossiness you had specular, diffuse and glossiness. The diffuse and glossiness bits are now in metallic and roughness instead.

So, previously, you worked either with specular/gloss or with metallic/roughness. Now you always have metallic/roughness, and optionally specular on top to enhance it. Metallic/roughness is not a fallback for specular.

[bbudzon]

I apologize if this issue has been misleading. I wanted to try to keep the conversation focused, but I think that is causing problems.

In truth, I'm trying to hit on a much broader point here than SpecGloss vs MetalRough workflows. I did feel this was the easiest to discuss, but perhaps we should dig deeper.

Generally speaking, many of the Khronos extensions to glTF are "post processing" to make transmission or memory on disk more efficient. Most of these types of extensions are "required"; meaning the program reading the file will be unable to read it without support for the extension. These types of extensions decrease interoperability as the program reading the file must support the extension in order to read it. So, restricting MVR so that these cannot be used increases interoperability, in my opinion.

In contrast, many of the Khronos extensions to glTF are optional; and it seems usually a direct result of authoring. MetalRough vs SpecGloss, clear coats, index of reflection, sheen, transmission, volumes, and even texture transforms. Because most of these extensions are "recommended" or "optional", the program reading the file can simply ignore the data. The impact on mesh quality will be noticed, but the impact on interoperability is negligible in my opinion.

So, at a much higher level, what I am suggesting is that we give the "no extensions allowed" requirement in GDTF a little more thought when handling MVR.

Should we encourage people to use KHR_materials_pbrSpecularGlossiness? No. Should we prohibit it such that a glTF containing it becomes an invalid MVR? I feel the answer here is also no. I see no issue with discouraging the use of optional extensions without prohibiting it. Required extensions, on the other hand, I feel should be explicitly prohibited.

So, in my opinion, one of MVR's goals should be interoperability and quality of meshes. Allowing things like draco mesh compression decreases interoperability where supporting extensions like sheen or clear coat increases mesh quality with no significant impact on interoperability.

[LxLasso]

In that sense I completely agree. And in my opinion the best way of achieving it would be to have a single glTF file be the main file / container and MVR be a glTF extension rather than the other way around.

But as long as MVR is the container and glTF is only used to pull in various bits and bobs (in a limited way with a few open issues such as how to deal with material repetition and asset animations) then I can see how restricting the extensions used might be a good thing.

[RichardTea]

The phrasing I had suggested was to limit the extensions that are permitted to be "required", so that data sources were free to add any glTF extensions they want, but data consumers are not required to understand them

[RichardTea]

I note that Vectorworks 2022 SP3.1 uses the "FB_ngon_encoding" extension, so does not comply with the "do not use extensions" requirement.

It is however not a "required" extension, so it appears a decision has been made that is not yet reflected in the document or public drafts.

[moritzstaffel]

@RichardTea Are you sure that this file comes from Vectorworks? We don't use this extension...

[RichardTea]

Yes, Vectorworks 2022 SP3.1 does include it in "extensionsUsed". Take a look.

I believe this extension merely flags that vertices have been triangulated in a certain way to allow the recovery of the original ngons, so has no effect on an importer that only cares about triangles.

As explained by the OP, it is perfectly reasonable for exporters to use "optional" glTF extensions to add extra data that isn't necessary to understand the file, but merely provides more context and perhaps better appearance if the renderer (and quality level) supports it.

[bbudzon]

Yes, @RichardTea. My proposal was that optional extensions such as sheen, clear coat, index of reflection, specular, and more should be allowed in the MVR and the spec updated accordingly. Given that these extensions are optional and do not prevent parsing of the glTF asset, I see no reason for an MVR to be consider "invalid" or "against the spec" for using these. In fact, allowing some of these optional extensions could improve the overall asset quality for programs that support reading them. And programs that cannot handle these extensions can simply ignore them.

I think it does make sense for MVR to mandate that no "required" extensions ever be used. The best example I can think of for this is draco mesh compressions. If a mesh is draco compressed and the reading program does not support it, then it cannot parse the mesh. While draco mesh compression may be beneficial for certain use cases, it doesn't have the "global interoperability" we are aiming for with MVR.

One last thing about VW and the "extensionsUsed". It appears that the library we are using handles things in a slightly odd fashion. Because the export library supports things like ngons and volumes, it lists them under "extensionsUsed". However, if you look at each individual mesh, no mesh exported out of VW will actually use these ngon or volumes extensions. At the time of this writing, the only extension that a mesh will actually use is the SpecGloss extensions. Do note, however, that a MetalRough material is provided as a fallback for those programs which do not support SpecGloss. So, in this sense, we adhere to SpecGloss being optional.

I hope this clarifies everything. All in all, my suggestion is that the MVR spec be updated to allow for optional extensions and only explicitly disallow any required extension that would prevent the asset from loading properly.

[moritzstaffel]

We need to do something about this in the next MVR Spec. MVR should allow the extension.