Add flipping mirrored normal map channels (RGB)

[tranek]

I posted this question on stackoverflow a few days ago and didn't get any replies in regards to Three.js so I am going to assume that this feature isn't implemented yet.

http://stackoverflow.com/questions/24208504/three-js-mirrored-normal-maps-flipped-channel

The issue is defined there but I will quickly summarize it. I have a model that is mirrored down the middle with a normal map. The normal map appears to have one channel (possibly red) that is flipped on the mirrored side of the model. I propose adding a new attribute to MeshPhongMaterial/Material that allows the developer to manually flip the channels of the mirrored side of the normal map.

For example, the three channels could be enumerated:

THREE.DefaultChannels
THREE.RedChannel
THREE.GreenChannel
THREE.BlueChannel

Then when defining the material, you would type:

var material = new MeshPhongMaterial()
... <material properties>
material.normal_mirrored_channel = THREE.RedChannel | Three.BlueChannel;
var mesh = new THREE.MorphAnimMesh( geometry, material );

This would flip both the red and blue channels on the mirrored side of the normal map using a bitwise OR on the enums with the default behavior as it is now. Obviously the names are just examples and something that better suits the conventions of Three.js should be used.

Might not be a bad idea to include the option to flip the non mirrored side as well if you need to compensate for tangent bias. For example, xNormal's default bias has the green channel flipped from 3DS Max.

[mrdoob]

I have a model that is mirrored down the middle with a normal map. The normal map appears to have one channel (possibly red) that is flipped on the mirrored side of the model.

Can you share a example to illustrate that?

[tranek]

Can you share a example to illustrate that?

Yes sir. Here's a link to a zip file of a working example.

https://www.dropbox.com/s/nnok2qmolqhmerx/threejsnormaltest.zip

It's a simple plane with some squiggles in it. I included a left and right original side model and tangent normal maps from both default xNormal and default 3DS Max. You can load whichever by commenting/uncommenting the code in this block:

// The model
// Original side on left, xnormal tangent normal map
loadModel("LP_plane_orig_left.js", "color.png", "LP_plane_xnormal_default_orig_left_normals.png");
// Original side on left, 3ds max tangent normal map
//loadModel("LP_plane_orig_left.js", "color.png", "LP_plane_max_default_orig_left_normals.png");
// Original side on right, xnormal tangent normal map
//loadModel("LP_plane_orig_right.js", "color.png", "LP_plane_xnormal_default_orig_right_normals.png");
// Original side on right, 3ds max tangent normal map
//loadModel("LP_plane_orig_right.js", "color.png", "LP_plane_max_default_orig_right_normals.png");

This is a screenshot of how I see it when loading it up: http://i.imgur.com/p1jZ6H6.png

I'm not sure why the plane isn't casting a shadow on the ground or why there's random shadowing on behind the plane on the ground, but those are questions for another topic.

[mrdoob]

Hmmm... Instead of making the shader more complex I think I would modify the texture instead. Meaning that you can draw the image into a canvas and invert the red values.

[tranek]

How would you only apply the texture to half of the model? Do you have it working on the example that I provided?

[mrdoob]

Uh? I don't see what applying a texture to half of the model has to do with inverting a chanel of the normal map...?

[mrdoob]

Ah wait, I see what you mean now... Is this a common thing? Is it for doing the normal map of one side of a character and reuse it in the other side with inverted channel?

[tranek]

Is this a common thing?

Yes sir it is very common in video games to save on texture space (higher quality textures!). In my own project, I'm using some existing video game assets that use this method.

Is it for doing the normal map of one side of a character and reuse it in the other side with inverted channel?

This is correct.

Someone replied to the stackoverflow post with a link to a thread on polycount detailing the issue (and the solution for Unreal Engine) http://www.polycount.com/forum/showthread.php?t=116922. Just in case that helps clear up any more confusion.

[WestLangley]

Instead of making the shader more complex I think I would modify the texture instead.

I agree.

[mrdoob]

Someone replied to the stackoverflow post with a link to a thread on polycount detailing the issue (and the solution for Unreal Engine) http://www.polycount.com/forum/showthread.php?t=116922. Just in case that helps clear up any more confusion.

Hmm, texture.wrapS = THREE.MirroredRepeatWrapping may be of some help.

[WestLangley]

Hmm, texture.wrapS = THREE.MirroredRepeatWrapping may be of some help.

It is my understanding that works for diffuse maps, but not for normal maps, which require additional modifications to the RGB texture data values on the mirrored half.

[tranek]

Hmm, you have texture.wrapS = THREE.MirroredRepeatWrapping.

Did you mean texture.wrapT ?

material.normalMap = THREE.ImageUtils.loadTexture(texture_normal);
material.normalMap.wrapS = THREE.RepeatWrapping;
material.normalMap.wrapT = THREE.MirroredRepeatWrapping;

I need that for both the normal and the diffuse textures since the UVs are flipped vertically when I import them into blender and thus into THREE.js format.

It is my understanding that works for diffuse maps, but not for normal maps, which require additional modifications to the RGB texture data values on the mirrored half.

This is pretty much the problem. I'm using it for the normal map and it is correct minus one color channel (supposedly red) is flipped on the mirrored half.

[tranek]

So are there any other ideas or is changing the shader the only option?

I've been poking around the shader code (I don't know much about shaders) and the only reference (that I've found so far) to the THREE.MirroredRepeatWrapping seems to end up on line 5938 in WebGLRenderer.js.

if ( p === THREE.MirroredRepeatWrapping ) return _gl.MIRRORED_REPEAT;

So if you set _gl.MIRRORED_REPEAT, webgl automagically handles the mirroring for you? Where in the shader code would someone put something to select the mirrored half of the model and swap a channel on the normal map?

Edit: Further research on the subject shows that multiplying the bitangent with the W component of the tangent seems to handle this automagically. I have no idea how that translates into the way that Three.js computes normals for the Phong shader. It looks like the "Normal map shader" may already do this?

http://en.wikibooks.org/wiki/GLSL_Programming/Blender/Lighting_of_Bumpy_Surfaces#Complete_Shader_Code

http://forum.unity3d.com/threads/symmetric-models-with-mirrored-normal-maps-shader-fix.65859/#post-420888

[shi-314]

Isn't it possible to create another material and set the normalScale.x value to -1?

[tranek]

Isn't it possible to create another material and set the normalScale.x value to -1?

Wouldn't you need a separate mesh for the mirrored half? That would create two unwelded vertices in the same spot all along the "mirrored" seam which would surely cause artifacting, no?

I tried it and it basically made it so the original side now has the flipped channel with the mirrored side being correct. So yes, you would need a separate mesh for the mirrored side.

I'm going to try switching from the Phong Material to this "Normal Map Shader" when I get a chance. I remember trying it before but it didn't work I think because I couldn't figure out how to set the wrapS and wrapT values for repeating and mirroring on T.

[shi-314]

You don't need two meshes but you need two materials assigned to the same mesh for the left and right side with different normaleScale values.

But yes it is not the most optimized way to do it :sweat_smile:

[tranek]

You don't need two meshes but you need two materials assigned to the same mesh for the left and right side with different normaleScale values.

How would you even assign the materials to only half of the mesh? It would be good to have at least one working solution as a backup regardless of how optimized it is or not!

[tranek]

So it turns out that I was using a really old version (like 1.2 vs 1.5 current) of the Blender exporter. It was from someone else's repository that allowed for multiple morph animations from the nla_tracks (https://github.com/madlion/three.js/tree/dev/utils/exporters/blender/2.62/scripts/addons/io_mesh_threejs). By switching to the Blender .js exporter in r67, it now correctly handles mirrored normal maps. The animations are broken, but I think that I can fix that eventually.

So it's a non-issue, my derp. Thanks everyone for contributing regardless!

[tranek]

As per @WestLangley request on the StackOverflow page, here's a live example of the Phong Shader vs the Normal Map Shader. The Phong Shader is on the left and the Normal Map Shader is on the right.

http://tranek.com/threejs-normal-test/

Not sure why the planes aren't casting a shadow on the floor plane. In my tests with other models, planes never cast shadows but enclosed models do cast the shadows. Question for another thread though.

[WestLangley]

@tranek

Your geometry has some problems, resulting in inverted tangents on the left half. I did not track that down. Use THREE.VertexTangentsHelper to see them.

If you use THREE.PlaneGeometry, both MeshPhongMaterial the normalmap shader work correctly.

geometry = new THREE.PlaneGeometry( 60, 120, 2, 1 );
geometry.computeTangents();

var material1 = new THREE.MeshPhongMaterial();
material1.map.wrapS = THREE.MirroredRepeatWrapping;
material1.map.wrapT = THREE.RepeatWrapping;
material1.normalMap.wrapS = THREE.MirroredRepeatWrapping;
material1.normalMap.wrapT = THREE.RepeatWrapping;
material1.map.repeat.set( 2, 1 );
material1.map.offset.set( 1, 0 );

var shader = THREE.ShaderLib[ "normalmap" ];
uniforms[ "tDiffuse" ].value.wrapS = THREE.MirroredRepeatWrapping;
uniforms[ "tDiffuse" ].value.wrapT = THREE.RepeatWrapping;
uniforms[ "tNormal" ].value.wrapS = THREE.MirroredRepeatWrapping;
uniforms[ "tNormal" ].value.wrapT = THREE.RepeatWrapping;
uniforms[ "uOffset" ].value.set( 1.0, 0.0 );
uniforms[ "uRepeat" ].value.set( 2.0, 1.0 );

three.js r.67

[tranek]

@WestLangley

The plane is mirrored down the middle. This is intentional to show a simple example of complex 3D models that are mirrored down the middle that exhibit this behavior that can't be replaced with a simple THREE.PlaneGeometry.

The problem is that the MeshPhongMaterial doesn't take into account the handiness or tangent.w like the normalmap shader does (hopefully I said that correctly).

I swapped out the plane with one of the actual models that I'm using. Please look at it again. http://tranek.com/threejs-normal-test/

[WestLangley]

@tranek Ah, yes. I get it.

MeshPhongMaterial does not use attribute tangents; it uses "derivative tangents" instead.

I expect there is an assumption in the Phong code about consistency the UV winding orders. Since your geometry is "mirrored", UVs are clockwise on one half and counterclockwise on the other half.

Fixing this -- if it is even possible -- will require some thinking... : - )

For reference, can you keep links in your post to both of your examples?

[tranek]

@WestLangley

Yes, I can keep that one site up for a little while (it was the same site for both examples, I just swapped out the model). You can even have the plane geometry and textures if you want since I just made those quickly from scratch just for this purpose.

I hope you can figure it out! Good luck!

[shi-314]

How would you even assign the materials to only half of the mesh? It would be good to have at least > one working solution as a backup regardless of how optimized it is or not!

Well your 3d artists can apply multiple materials to an object. What I do is iterate over the material names from the export. If you apply multiple materials to an object e.g. in Maya and export it as an OBJ your Mesh will have a MeshFaceMaterial which is just a list of materials. You can iterate over the materials and replace them by their name with the material you want.

[tranek]

Well your 3d artists can apply multiple materials to an object. What I do is iterate over the material names from the export. If you apply multiple materials to an object e.g. in Maya and export it as an OBJ your Mesh will have a MeshFaceMaterial which is just a list of materials. You can iterate over the materials and replace them by their name with the material you want.

Ah I see. That makes sense. Thank you!

[SonofDanube]

Hi Guys, appologies if this doesn't belong fully to this thread, it's on behalf of rendering of mirrored normal maps in 3DsMax viewport, and someone can be looking for this information also here:

In case anyone comes across problem that 3DsMax (2012-2015) displays one half mirrored normal maps flipped (verticaly in my case), baked in xnormal or 3dsmax, here is the solution: When loading normal map into material editor, there is small field in the loading dialog "gamma", and it's checked as "automatic" (which means 2,2), change it to "override" (1,0) there, and mirrored normal maps will be displayed correctly.

[pixeldamage]

Sorry to necro an old thread but I'm having the same mirrored-normal map issue.

I'm using FBX2013 from 3dsmax and exporting with tangents/binormals included. Setting texture.wrapS = THREE.MirroredRepeatWrapping on the normal map shows the inverted issue (which I believe mentioned above is because the red channel still needs flipping on that part).

Is there an easy way to flip the red channel of the mirrored half without using extra texture lookups? The polycount/UDK method listed above requires doing this manually in the shader (as well as flipping the mirrored UVs back to the correct way prior to exporting).

[WestLangley]

@pixeldamage As I stated in your SO question, go back to using unmirrored UVs. three.js does not utilize geometry tangents in its built-in shaders. You can file an enhancement request to honor MirroredRepeatWrapping in normal maps if you want, but there has been minimal demand for that feature -- especially since there is a workaround.

[bhouston]

There is a solution to this problme outlined in this comment, we should just make a PR and fix this permanently:

https://github.com/mrdoob/three.js/issues/7094#issuecomment-376944635