glTF 2.0

[lilleyse]

This has initial support for gltf 2.0.

I would like some input on my approach on converting traditional diffuse/specular materials to pbr metallic-roughness materials. These are the basic steps:

• Specular intensity is extracted from the specular color and treated as the metallic factor.
• Specular shininess is typically an exponent from 0 to 1000, and is converted to a 0-1 range as the roughness factor.
• Diffuse stays as is

Also I have one comment left in the code that I'm wondering about. Is there a convention for lighting models when they don't contain normals? Should pbrMetallicRoughness be left undefined and let the implementation figure it out? Right now I do keep the pbr material but set all the values to either black or 0.0, and set the emissive color instead. I'm wondering what different glTF 2.0 viewers might do with a material like this.

To-do:

• [x] More manual testing. Fix tests.
• [x] Figure out how to get pngjs to write out a 24-bit png with their sync packer. May require a PR there.

[pjcozzi]

@sbtron @bghgary @lexaknyazev @mlimper @cedricpinson @erich666 do you have any input on this approach to convert diffuse/specular materials to PBR metal roughness? See https://github.com/AnalyticalGraphicsInc/obj2gltf/pull/67#issue-222887037

[emackey]

My $0.02, I don't agree with the traditional specular intensity being used as PBR metalness. Traditional diffuse/specular (non-PBR) models never look metallic, so I think it's fine to hard-code metalness to zero for such conversions.

The roughness value comes from both specular intensity and shininess. When specular intensity is very low or zero, roughness should be very high, regardless of the shininess setting. When specular intensity is above near-zero, then you have to look at the shininess factor to determine the roughness. I think generally the higher shininess exponents correspond to smoother surfaces, and lower ones to rough surfaces. I would have to play around with some sample models to see what looks visually similar, though.

[xelatihy]

The approach you mention is quite lossy. Things will look very poor. We has this discussion some time ago when I was pushing the diffuse-specular instead, but I got voted out.

Simple thing to do:

1. convert to PBR diffuse-specular extension
2. from that loossily convert to PBR metal-roughness
3. for the specular exponent, use the conversion suggest in the EGSR paper on GGX

Example code (not finished) in yocto-gl.

[bghgary]

@xelatihy Can you point to a link for the EGSR paper for the conversion you mentioned?

I did some experiments with this when we were talking about PBR parameter sets. For context, here is the thread. I would recommend doing what @xelatihy said in 1 and 2, not sure about 3. The conversion from spec-gloss to metal-rough is lossy and none of them are perfect.

The live demo of my conversion is here. It is based on the math described here. It's documented in the specs here and here. There is also a note about conversion in the spec-gloss extension appendix. The conversion is not perfect and materials sometimes end up more metallic than they should be. I've been meaning to document how I ended up with the conversion equations, but haven't gotten around to it. Let me know if documenting it will be useful.

We have been using this conversion to convert spec-gloss to metal-rough for a few of our models in our testing repo. The Avocado and BarramundiFish are examples of the conversion.

Spec-Gloss:

• https://sbtron.github.io/BabylonJS-glTFLoader/?model=Avocado&folder=glTF-pbrSpecularGlossiness
• https://sbtron.github.io/BabylonJS-glTFLoader/?model=BarramundiFish&folder=glTF-pbrSpecularGlossiness

Converted to Metal-Rough:

• https://sbtron.github.io/BabylonJS-glTFLoader/?model=Avocado
• https://sbtron.github.io/BabylonJS-glTFLoader/?model=BarramundiFish

You can toggle back and forth between the two workflows with the drop-down.

[bghgary]

Is there a convention for lighting models when they don't contain normals?

We had a small discussion about calculating normals https://github.com/KhronosGroup/glTF/commit/fb8b472701bf54fc2dd80b1ade2dd312f42113ba#commitcomment-21728860. We ended up saying that flat normals will be used if they are not specified in the asset.

This is specified in an implementation note in https://github.com/KhronosGroup/glTF/tree/2.0/specification/2.0#meshes

[pjcozzi]

There's some great discussion on twitter, https://twitter.com/pjcozzi/status/855067965309558784

(also above, https://github.com/AnalyticalGraphicsInc/obj2gltf/pull/67#issuecomment-295777569)

[xelatihy]

BTW, for the PBR material I should also mention that you should look at the OBJ illum property.

[emackey]

This is great discussion of the ideal math by @xelatihy and @bghgary. But allow me to offer a completely different point of view, from a typical artist-type user attempting to use this obj2gltf tool. I feel like I have a half-decent understanding of the current workflow for at least some artists who are using OBJ and PBR, after talking to a few in person, watching many online tutorials, and using some of the tools myself to create some sample models (via Collada since this OBJ tool is not yet up to the task). The current workflow goes something like this:

1. Start creating the model in a package such as Modo, Blender, etc. Much of the focus at this stage is on geometry, normals, and a reasonable UV map. Some tools at this stage have some awareness of PBR (I think Modo can show PBR in its viewport, and Blender has plans to do likewise sometime next year), but most modeling tools do not have the final say on PBR. Generally the model only has "placeholder" textures at this point, or has an "ID" map (one solid color per material, just to indicate material locations on a model).

2. The model is exported to an intermediate format, such as Collada, FBX, or OBJ. Since this PR is for the OBJ converter, we'll assume OBJ export here. Since we're assuming OBJ format, we know that no actual PBR information was written in step 1, since OBJ does not support that. This is common practice.

3. A different set of tools come into play to create a full set of PBR textures for the OBJ file. This could be Substance Painter, Marmoset Toolbag, or even just Photoshop if the artist has some way to preview what the OBJ looks like when combined with PBR textures. This could even mean loading the model into an engine like Unity 3D, and interactively assigning PBR textures there.

The important thing to understand here is the situation the artist is placed in by the end of step 3: The OBJ still exists, un-modified from step 2. Likewise, some PBR textures now exist, created in step 3 to pair up with the UV map that's baked into the OBJ. But the OBJ doesn't call out the PBR textures by filename, as there are several of those per-material, and they didn't exist yet when the OBJ was created.

In fact, in the particular case of a truck model I got from one of our 3D model artists here (Scott), the texturemap name baked into the OBJ is unreadable ("F:\Scott\GroundVehicle\groundvehicle.png doesn't exist"). He sent me the OBJ and a small stack of PBR textures exported from Substance Painter, for two different materials in this particular OBJ. The PBR maps are named like this, which is quite typical:

GroundVehicle_body_BaseColor.png
GroundVehicle_body_MetalRough.png
GroundVehicle_body_Occlusion.png
GroundVehicle_body_Normal.png
GroundVehicle_wheels_BaseColor.png
GroundVehicle_wheels_MetalRough.png
GroundVehicle_wheels_Occlusion.png
GroundVehicle_wheels_Normal.png

So for him to use this obj2gltf tool himself, he would need a way to specify the OBJ file, and then specify all eight of these PBR textures, and have the tool understand which texture was assigned to what PBR channel of which material in the OBJ. Plus, the tool would need to disregard the actual texture filename that was specified in the OBJ file itself, as that was just a work-in-progress, non-PBR texture name.

The command-line may not be the right approach here. Of course artists have never liked command lines, but in this case it's particularly egregious as the number of files involved increases by 4 per material.

I think what artists like Scott need would be more along the lines of a GUI that can load an OBJ, and then allow interactive assignment of textures to PBR channels per material, ideally with a live preview window, and then save the result to glTF 2.0. Actually I have some hope that my vscode extension will someday fill this role, but currently vscode extensions don't have a way to open file dialogs (there's an issue filed) and that puts a crimp in those plans for now.

It may be fair to say that interactive assignment of PBR maps is out-of-scope for obj2gltf for now. If that's the case, then some replacement priorities bubble up:

• obj2gltf should be easy to embed and use within larger tools, such as the interactive one I describe.
• obj2gltf should be quite tolerant of OBJ textures that don't exist, as the textures there may be completely meaningless. Maybe even put a flag to disregard any textures called for by the OBJ (I don't have an F drive on my machine for example, and I don't want the tool reading from any filenames it found in the OBJ file). Keep in mind that the contents of the OBJ pre-date the artists's own efforts to create PBR textures for the model.
• obj2gltf should correctly preserve geometry, normals, and especially preserve UV maps even if the texture is not found or intentionally disregarded.
• obj2gltf should create stubs for the materials with the correct material names assigned. These could have a simple base-color map assigned from the OBJ's texture filename, unless the user opted to disregard OBJ textures, in which case these would just be named empty material stubs.

With these qualities, obj2gltf could be used as the initial model load stage of a future interactive tool for assigning PBR textures to glTF files. For users who aren't using PBR, they should just stick to KHR_material_common, or the tool could attempt to do some sort of up-conversion via the spec/gloss PBR workflow as described by other participants here. But I see this as lower priority, mostly because the result can't compete with actual PBR assets like Scott's truck. OBJ simply doesn't hold the full suite of PBR texturemaps, and there's no authoring tool that tries to squeeze PBR into OBJ itself, so there's limited value in attempting to extract it back out.

I would much rather see obj2gltf focus itself on enabling professional PBR, and not attempting to up-convert from non-PBR. All of this is just my personal opinion of course :)

[donmccurdy]

I think what artists like Scott need would be more along the lines of a GUI that can load an OBJ, and then allow interactive assignment of textures to PBR channels per material, ideally with a live preview window, and then save the result to glTF 2.0.

I think this could be supported in the gltf-viewer I've been working on (demo, source). Potential workflow:

1. Import .gltf asset without PBR maps.
2. Select a mesh from scene graph.
3. Drag-and-drop PBR maps and assign.
4. Preview.
5. Export .gltf result.

I would have a few more questions before trying to implement something like that, probably more appropriate for a separate issue on the glTF repo.

In any case, +1 from me on trying to avoid up-conversion to PBR in obj2gltf.

[pjcozzi]

It sounds great to have the flexibility that @emackey suggests (contributions welcome, of course!); however, obj2gltf will still need some reasonable default way to create glTF models without the user having to think about the materials. like today's workflow. This can be built on top of the flexibility, but we are making the barrier-to-entry for glTF too high if we go without it.

[donmccurdy]

obj2gltf will still need some reasonable default way to create glTF models without the user having to think about the materials. like today's workflow.

I agree that we need good defaults, and an OBJ going through obj2gltf should yield a similar-as-possible glTF asset. But why should obj2gltf up-convert from phong to PBR by default, when phong can be defined simply with the KHR_materials_common extension? That conversion seems more appropriate for gltf-pipeline than obj2gltf, in my opinion.

[pjcozzi]

If KHR_materials_common becomes widely supported, that could be the right approach, but as of now, the extension is not finished, ratified, or widely implemented, so we still need a mapping to PBR.

[emackey]

so we still need a mapping to PBR.

Perhaps we can make this tool align with what COLLADA2GLTF is doing? Rob (@lasalvavida) describes that process in a comment here.

[lasalvavida]

You can find the C++ implementation for this here.

In general: diffuse(Color) -> baseColorFactor diffuse(Texture) -> baseColorTexture emissive(Color) -> emissiveFactor emissive(Texture) -> emissiveTexture ambient(Texture) -> occlusionTexture bump -> normalTexture

If we have pbrSpecularGlossiness: diffuse(Color) -> diffuseFactor diffuse(Texture) -> diffuseTexture specular(Color -> specularFactor specular(Texture) -> specularGlossinessTexture shininess -> glossinessFactor

And then metallicFactor, roughnessFactor, and metallicRoughnessTexture all have to be specified via command-line as there is no direct mapping.

[lasalvavida]

Also if anyone has any suggested changes or improvements on this approach I'm happy to hear them; I'm sure many of the people reading this know a ton more about rendering than I do, and may be able to help make this mapping more robust

[emackey]

And then metallicFactor, roughnessFactor, and metallicRoughnessTexture all have to be specified via command-line as there is no direct mapping.

Actually, for metal/rough models, the ambient texture might actually be a combination of occlusion (R), roughness (G), and metalness (B) in a single map. I could imagine a command-line switch to enable usage of such a combined texture from the ambient channel in both the occlusionTexture and metallicRoughnessTexture channels (using the same texture index for both).

[pjcozzi]

Adding @bghgary who has been thinking about this.

[lilleyse]

The code is still not 100% ready, but I've pretty much converged on the workflow for this tool.

The command line now has three options:

• metallicRoughness - The values in the mtl file are already metallic-roughness PBR values and no conversion step should be applied. Metallic is stored in the Ks and map_Ks slots and roughness is stored in the Ns and map_Ns slots.
• specularGlossiness - The values in the mtl file are already specular-glossiness PBR values and no conversion step should be applied. Specular is stored in the Ks and map_Ks slots and glossiness is stored in the Ns and map_Ns slots. The glTF will be saved with the KHR_materials_pbrSpecularGlossiness extension.
• materialsCommon - The glTF will be saved with the KHR_materials_common extension. Good for assets with traditional materials and engines that support this extension.
• TODO : add command that outputs in KHR_technique_webgl

If none of these are supplied, the material is converted from traditional to metallic-roughness PBR.

The alternative of passing in metallic/roughness/specular/glossiness values/textures via command line seemed like too much work and maintenance, particularly when multiple materials are involved. This method keeps it simple, but requires that the obj is ready-to-go. As @emackey and @donmccurdy discussed a gui may be more suitable for the task of connecting pbr maps to fix the possibly irrelevant .mtl file.

I've decided to keep the traditional->metallic-roughness conversion pretty simple. I started to watch a video about how an artist would convert traditional textures to pbr, and there's too much magic involved to code a proper conversion in here. Along those lines I couldn't find a good way to convert traditional to specular-glossiness, they sound the same but seem to represent completely different concepts. Looking at some sample PBR values here the traditional diffuse would not map well to the specular-glossiness diffuse.

So I mainly went with @emackey's suggestion and decided that this tool should just produce good enough values for simple models that are exported at default settings from blender and other programs, but should not be a full blown converter. For example I don't bother converting the specular/shininess textures to PBR because it just feels beyond the scope.

The new conversion looks like:

• diffuse -> base color
• specular intensity and specular shininess -> roughness
• metallic -> 0

[emackey]

@lilleyse This all sounds great. One question though (I haven't looked at the code changes yet), why do you use both map_Ks and map_Ns? Are you reading those maps, combining channels, and outputting a new map made of combined channels? What if the user already has the maps combined with the channels?

And perhaps most importantly, what if the user has a map with combined occlusion (R), roughness (G), and metalness (B) as a single texture already, do you support that? That should be the most common case at least for coming out of Substance Painter (and Unity I think) and going to glTF.

[lilleyse]

Are you reading those maps, combining channels, and outputting a new map made of combined channels? What if the user already has the maps combined with the channels?

Right now the assumption is that no textures are packed together. The converter will pack metallic, roughness, and occlusion maps together.

[emackey]

The converter will pack metallic, roughness, and occlusion maps together.

I'm sure there are situations where that could be useful. But of course "classic" OBJ models don't have these new-fangled PBR maps to start with, so only PBR-aware models would ever need this channel re-packing. Most tools that produce PBR maps will produce them pre-packed, either with configurable mapping or with their own hard-coded mapping. But generally these tools do not produce a pile of loose channels like this in my experience. So having the OBJ converter do this automatically, by default, strikes me as problematic.

There should at the very least be an option to enable or disable texture repacking (vs just embedding texture names in the glTF and not loading or altering the textures themselves). Probably this should be off-by-default, as neither classic OBJs nor PBR OBJs typically need this kind of remapping.

[lilleyse]

I'm not super familiar with all the exporters, do they generally follow the same packing conventions?

[emackey]

I'm familiar with Substance Painter. It has a number of preset export configurations to pre-package the channels for Unity or Unreal or a number of other targets. It's easy to create a preset for glTF that pre-packages spec with gloss, or pre-packages occlusion/rough/metal together. And generally, Substance Painter users would want their glTF to use the exported textures directly, with no remapping.

I'm not as familiar with what other PBR tools do. Reading through some Marmoset Toolbag documentation, it looks like Toolbag may expect separate channels to come in through separate images, so, that sounds more aligned with your automatic channel remapping.

Maybe you could just look for map_Ks to exist in the absence of map_Ns, and treat that as the indication that the channels have already been packed (IE, use the supplied map_Ks directly, don't zero out a channel from it). If both exist, then you could reason that a remapping is needed to combine them.

[donmccurdy]

It's easy to create a preset for glTF that pre-packages spec with gloss, or pre-packages occlusion/rough/metal together.

I believe that Substance Painter's UE4 occlusionRoughnessMetallic preset would already be directly compatible with glTF (details)

[emackey]

I believe that Substance Painter's UE4 occlusionRoughnessMetallic preset would already be directly compatible with glTF

I just looked, and yes.....mostly. Except for one tiny detail:

Oh no! It's a DirectX normal map, not an OpenGL normal map. From my own experiments, I believe the only difference is the intensity of the green channel is negated.

I'm not a betting man, but if I were, I'd be betting my entire budget that this little wrinkle is the primary cause of donmccurdy/three-gltf-viewer#10.

Do we know, officially, if the green channel of normal maps is supposed to be positive or negative in glTF 2.0? I may need to open an issue in the main repo to ask.

[mlimper]

Hi,

Do we know, officially, if the green channel of normal maps is supposed to be positive or negative in glTF 2.0? I may need to open an issue in the main repo to ask.

yes, it would be cool to clarify this. With our current export pipeline we can directly export textures for usage with the BabylonJS Web viewer. If we want to upload on Sketchfab, however, we'll have to go to the Sketchfab backend's 3D settings and uncheck "Flip green (-Y)" there, otherwise the normal maps wouldn't be interpreted correctly:

We should probably spell out explicitly in the 2.0 spec which convention we want to follow, not sure if that was already done within the last draft, or if we need to open an issue and figure that out.

[emackey]

I plan to open an issue, with diagrams. I'll link from here when done. Apologies to obj2gltf for going off-topic.

[emackey]

Normal map problems moved to KhronosGroup/glTF#952. Sorry for the disruption here.

[pjcozzi]

Bump @lilleyse

[lilleyse]

@emackey I added the ability to specify texture paths on the command line, which will help out with the conversion workflow with Substance Painter, etc. The options are a bit verbose, but they are:

• --metallicRoughnessOcclusionTexture - Path to the metallic-roughness-occlusion texture used by the model, where occlusion is stored in the red channel, roughness is stored in the green channel, and metallic is stored in the blue channel. This may be used instead of setting texture paths in the .mtl file. The model will be saved with a pbrMetallicRoughness material.
• --specularGlossinessTexture - Path to the specular-glossiness texture used by the model, where specular color is stored in the red, green, and blue channels and specular glossiness is stored in the alpha channel. This may be used instead of setting texture paths in the .mtl file. The model will be saved with a material using the KHR_materials_pbrSpecularGlossiness extension.
• --occlusionTexture - Path to the occlusion texture used by the model. This may be used instead of setting texture paths in the .mtl file. Ignored if metallicRoughnessOcclusionTexture is also set.
• --normalTexture - Path to the normal texture used by the model. This may be used instead of setting texture paths in the .mtl file.
• --baseColorTexture - Path to the baseColor/diffuse texture used by the model. This may be used instead of setting texture paths in the .mtl file.
• --emissiveTexture - Path to the emissive texture used by the model. This may be used instead of setting texture paths in the .mtl file.

[lilleyse]

As part of the cleanup here gltf-pipeline is no longer a dependency, so the options compress, optimize, generateNormals, optimizeForCesium, ao, and bypassPipeline are no longer included.

[lilleyse]

I tested a variety of models and options, this is ready to review now.

@likangning93 would you be able to review this?

[lilleyse]

@likangning93 I think I address everything except https://github.com/AnalyticalGraphicsInc/obj2gltf/pull/67#discussion_r129646700. I'm going to work on that a bit later.

[donmccurdy]

I just tested this on a bunch of simple OBJs I had lying around. Looks good and nice work!

[pjcozzi]

@lilleyse what is the plan here? 😄

[lilleyse]

Still finishing up the tests after reorganizing how materials are loaded. It won't be read by the glTF bof, but is close.

[pjcozzi]

Rock on and merge when ready!

[lilleyse]

@likangning93 this is now ready for final review. I got a bit carried away and went on a cleaning/reorganization spree. Sorry about the large diff yet again. I believe it is final

The major changes since before are:

• No longer saves the converted glTF from within the API, instead returns the JSON or glb and the caller is responsible for saving it. Saving separate resources is still supported - but requires either passing an outputDirectry or a custom writer callback.
• Writing glb's no longer has the buffer size limitation
• Materials, PBR texture creation, and handling of overriding textures is now all in loadMtl. This keeps createGltf a lot leaner.
• Consistent about naming for image/texture. Texture is now used throughout, except when referring to gltf.images.

[mramato]

No longer saves the converted glTF from within the API, instead returns the JSON or glb and the caller is responsible for saving it. Saving separate resources is still supported - but requires either passing an outputDirectry or a custom writer callback.

Doesn't have to be this PR, but eventually we should support a stream-based implementation so that you simply write objects to a user-supplied stream, that way you could handle multi-file output without having to write to a specific directory (think DB import)

[lilleyse]

@likangning93 merged in master, all ready now.

[likangning93]

Thanks @lilleyse!