[BUG-5975] Normal map rendering issue when UV island tangent basis has angular difference and mesh is smooth shaded

[sl-service-account]

Steps to Reproduce

Issue can be reproduced with any (typically organic model) mesh that

• is smooth shaded across uv island borders (one rendered vertex in uv seam, instead of two vertices like in hard edge)
• has more than one uv-island
• uv layout is such that there is some angular difference between islands, and vertex tangent basis across the uv border has difference.

• Normal map (1024 x 1024, openGL, x+ y+ z+, 12 px edge padding) baked from Hi-Poly with any widely used bakers: Blender, XNormal or ZBrush

  Attached images and files will clarify this issue. In the images, there is some specular effect added to make seam more visible in static image. Normal map is however the cause of this according to tests.

  Test number 1 is a extract from a human torso.

  Test number 2 is a SL default avatar. Right side of SL avatar was deleted to avoid mirrored and overlapped uvs in arm area.

  In both cases normal map was baked from subdivided hi-poly mesh.

  Actual Behavior

  Normal map renders wrong across uv border seam of smooth shaded mesh, when certain typical conditions are met. The issue is not visible in specular or diffuse channel, only normal map channel makes this visible in render.

  When uv-islands of a smooth mesh are rotated so that there is a difference in tangent basis between vertices across the seam -> rendered face normals start to point wrong direction. The larger the tangent basis difference, the stronger visible seam is rendered.

  Expected Behavior

  Several tests has been performed using non-SL shaders. Marmoset Toolbag, XNormal 3D view, Blender GLSL Shading and Nvidia FX Composer 2.5 render correctly, with no seam. So, this could be a bug in SL shader.

  Other information

Attachments

• NM for test_dae.tga 2014-05-10 08:30:39
• normal_map_test.dae 2014-05-10 08:30:39
• test_number_1_uv.png 2014-05-10 08:30:39
• test_number_1.png 2014-05-10 08:30:39
• test_number_2_uv.png 2014-05-10 08:30:39
• test_number_2.PNG 2014-05-10 08:30:39

Links

Duplicates

• [ ] https://github.com/secondlife/jira-archive/issues/5729
• [ ] https://github.com/secondlife/jira-archive/issues/5734

  Related

• [ ] https://github.com/secondlife/jira-archive/issues/5968

Original Jira Fields

| Field | Value | | ------------- | ------------- | | Issue | BUG-5975 | | Summary | Normal map rendering issue when UV island tangent basis has angular difference and mesh is smooth shaded | | Type | Bug | | Priority | Unset | | Status | Needs More Info | | Resolution | Unresolved | | Reporter | Jake Koronikov (jake.koronikov) | | Created at | 2014-05-10T12:30:39Z | | Updated at | 2021-06-09T16:48:46Z | ``` { 'Business Unit': ['Platform'], 'Date of First Response': '2014-05-10T08:15:25.637-0500', "Is there anything you'd like to add?": '\r\nIssue can be reproduced with any (typically organic model) mesh that\r\n- is smooth shaded across uv island borders (one rendered vertex in uv seam, instead of two vertices like in hard edge)\r\n- has more than one uv-island\r\n- uv layout is such that there is some angular difference between islands, and vertex tangent basis across the uv border has difference.\r\n- Normal map (1024 x 1024, openGL, x+ y+ z+, 12 px edge padding) baked from Hi-Poly with any widely used bakers: Blender, XNormal or ZBrush', 'System': 'SL Viewer', 'Target Viewer Version': 'viewer-development', 'What just happened?': 'Normal map renders wrong across uv border seam of smooth shaded mesh, when certain typical\r\nconditions are met. The issue is not visible in specular or diffuse channel, only normal map channel makes this visible in render.\r\n\r\nWhen uv-islands of a smooth mesh are rotated so that there is a difference in tangent basis between vertices across the seam -> rendered face normals start to point wrong direction. The larger the tangent basis difference, the stronger visible seam is rendered.\r\n', 'What were you doing when it happened?': 'Attached images and files will clarify this issue. In the images, there is some specular effect added to make seam more visible in static image. Normal map is however the cause of this according to tests.\r\n\r\nTest number 1 is a extract from a human torso. \r\n\r\nTest number 2 is a SL default avatar. Right side of SL avatar was deleted to avoid mirrored and overlapped uvs in arm area.\r\n\r\nIn both cases normal map was baked from subdivided hi-poly mesh. \r\n', 'What were you expecting to happen instead?': 'Several tests has been performed using non-SL shaders. Marmoset Toolbag, XNormal 3D view, Blender GLSL Shading and Nvidia FX Composer 2.5 render correctly, with no seam. So, this could be a bug in SL shader. ', } ```

[sl-service-account]

Drongle McMahon commented at 2014-05-10T13:15:26Z, updated at 2014-05-10T21:57:16Z

I had described an effect that I thought was related to this jira, but turns out to be mainly because of the incorrect value used in the inbuilt "Blank" normal map. I have therefore deleted the comment as it does not add to this jira, merely provided another example.

[sl-service-account]

Geenz Spad commented at 2014-10-01T19:13:40Z

What I suspect is happening here is how the viewer generates a special set of coordinates called tangents.

Tangents are required for normal mapping to function properly. What you're seeing here looks to me like the tangents your normal map was baked with aren't lining up with what the viewer is generating. This isn't surprising, as most 3D modeling packages have their own, often times proprietary, way of calculating tangents on a surface. The method that the viewer uses (or at least, what the viewer's implementation is based off of) can be found here: http://www.terathon.com/code/tangent.html

What this can result in is edge discontinuities like you're experiencing in SL. As for other applications like Toolbag, those applications actually import the tangents that were generated by the 3D package you exported from to ensure that normal maps all look correct when viewed within those applications.

Second Life does not do this for the sake of file size, and instead generates them on the fly for meshes and most other geometry.

You likely won't be seeing a fix for this any time soon, as it basically requires that the SL mesh format support tangents which it currently does not.

[sl-service-account]

Alexa Linden commented at 2015-03-24T16:33:16Z

Good morning all.

Can you please try http://automated-builds-secondlife-com.s3.amazonaws.com/hg/repo/simon-ll-viewer-lion/rev/299731/index.html and let me know if you're still able to reproduce this issue?

Thanks!

[sl-service-account]

Whirly Fizzle commented at 2015-03-24T17:19:20Z

Bug still reproduces on Second Life 3.7.26 (299731) Mar 11 2015 03:37:11 (Second Life Test)

Lion: http://i.imgur.com/HeMw64w.jpg Default release: http://i.imgur.com/OuFkH9t.jpg

Second Life 3.7.26 (299731) Mar 11 2015 03:37:11 (Second Life Test)
Release Notes

You are at 91.8, 113.0, 21.8 in Testylvania Sandbox located at sim10444.agni.lindenlab.com (216.82.51.150:13018)
SLURL: http://maps.secondlife.com/secondlife/Testylvania%20Sandbox/92/113/22
(global coordinates 332,636.0, 306,289.0, 21.8)
Second Life Server 15.03.11.299741
Retrieving...

CPU: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz (3491.96 MHz)
Memory: 16268 MB
OS Version: Microsoft Windows 7 64-bit Service Pack 1 (Build 7601)
Graphics Card Vendor: NVIDIA Corporation
Graphics Card: GeForce GTX 750/PCIe/SSE2

Windows Graphics Driver Version: 9.18.0013.4788
OpenGL Version: 4.5.0 NVIDIA 347.88

libcurl Version: libcurl/7.38.0 OpenSSL/1.0.1h zlib/1.2.8
J2C Decoder Version: KDU v7.0
Audio Driver Version: FMOD Ex 4.44.31
Qt Webkit Version: 4.7.1 (version number hard-coded)
Voice Server Version: Vivox 4.6.0009.20030

Built with MSVC version 1600
Packets Lost: 0/1,850 (0.0%)

[sl-service-account]

Jake Koronikov commented at 2015-03-25T16:10:48Z

Hi! I'm just confirming same as Whirly Fizzle above: reproducable using the test version.

[sl-service-account]

Saeros Linden commented at 2017-07-07T00:44:23Z

Our apologies, it appears this jira has been a victim of a spammer. We’re cleaning up the offending comments, sorry for the mess!

[sl-service-account]

Vir Linden commented at 2019-01-03T21:23:41Z

At least in some cases this seems to be an issue with the normal maps. If there are example normal maps which you are sure (a) are not discontinuous but (b) still show the bug, could you attach them here or otherwise link to them?

[sl-service-account]

Whirly Fizzle commented at 2019-08-07T16:08:37Z

Hiya Vir,

Possibly the same bug filed at BUG-227448, with repro dae & normal map.

[sl-service-account]

Whirly Fizzle commented at 2019-08-07T16:16:19Z

BUG-227442 is possibly also the same bug (has repro).

[sl-service-account]

Kyrah Abattoir commented at 2020-01-08T15:42:48Z

My examples on https://jira.secondlife.com/browse/BUG-227448 Uses a blank normal map

[sl-service-account]

Kyrah Abattoir commented at 2020-04-20T15:22:05Z

Any news on this problem?

[sl-service-account]

Naiman Broome commented at 2021-01-20T16:43:32Z

I am having this exact problem , is this beeing worked on ?

[sl-service-account]

Ptolemy Linden commented at 2021-05-18T21:48:32Z

I have this fixed in a local branch for BUG-227448 which should fix this issue as well.  It should be available in LMR #6 when it releases.

If anyone has more test models that demonstrate this bug please feel free to add them to this Jira so I can verify I haven't missed any edge cases.

Thanks

[sl-service-account]

Kyrah Abattoir commented at 2021-05-19T13:04:39Z

Have you tested it with the object in https://jira.secondlife.com/browse/BUG-227448 ?

[sl-service-account]

Ptolemy Linden commented at 2021-05-21T14:46:58Z

Yes, as previously mentioned, I have this working for the sphere in BUG-227448.

This week as I collecting models with various normalmaps I was finally able to track down 3 really good edge cases. Unfortunately upon further inspection I am able to see the texture seams with the torso example here.

I am currently pursuing 2 solutions to this. I'll update when I have more news to share.

[sl-service-account]

Kyrah Abattoir commented at 2021-05-21T15:24:47Z

You're probably very busy, but can you tell us what was causing this when you have a moment?

[sl-service-account]

Ptolemy Linden commented at 2021-05-24T18:19:06Z, updated at 2021-05-24T18:19:34Z

I'm OOO (out-of-office) for the next few day but yes, I can summarize.

TL:DR; The "texture seam" / lighting "edge" comes about if:

• a model has discontiguous use of a normal map, and

• the calculation for the normal (based on the normal map being used) in rendering doesn't match the same algorithm as the content creation application used to initially create ("bake") the normal map.

  The per pixel normals end up "different enough" and thus the lighting is not contiguous causing the "harsh edge" as seen in the torso sample here.

  The long version:

  Normally, this isn't an issue since (most?) objects sample from a "contiguous" region in the normal map. Organic models tend to have discontinuities though and thus you may be more likely to see these "texture seams" in these cases.

  Here is an analogy:

  Q. You are at the north pole. Which "direction" would you pick to walk south?

  A. There are infinite directions on the surface you can pick. In Mathematics we call edge case a "singularity" – where you have multiple answers and they are all "correct".

  However, it might be beneficial to pick one preferred tangent to the surface over another one. We can use a polygon's edges to define one of tangents to the surface. The cross product of the normal and tangent produces a 3rd bi-tangent. This is part of the TBN (tangent, bi-tangent, normal) calculation which is used to interpolate a "final" normal across a surface for lighting calculations. One wrinkle is that the normal, tangent, and bi-tangent "look" like a normal 3D vectors but technically they are not – they are "co-vectors". They behave _slightly _different. The model in this Jira (and in BUG-227448) demonstrate the edge case where they behave slightly differently and thus produce the rendering artifact. Hence the "texture seam" bug.

  Another part of the problem is that there is no "one right way" to interpolate the normals across a face of a polygon. The graphics community has standardized on some methods and 99% of the time there are no issues. However, once we have a normal map – we have an edge case – similar to the "walk south from the north pole" analogy above. Keep in mind that a normal map is a visual approximation of more detailed geometry. "Good enough" of the reconstruction of a normal that approximates the visual appearance of the high detail geometry can be done in different ways! It is, and was, easy to miss how this final normal should be calculated. As CPUs and now GPUs, have gotten faster, we can do a better job of more accurately approximating this "final normal."

  Without going into specifics, there have been various algorithms over the years: Eric Lengyel's original 2001 tangent calculation (which is what Second Life uses), 3DS Max, Maya, mikktspace, NVMeshMender, etc. that all do "their own thing" as it is not easy to fully understand the problem that normals, tangents, and bi-tangents are technically co-vectors and shouldn't be treated as vectors.

  Normally (pardon the pun) you don't see the minor issue of a texture seam unless you are specifically looking for it AND the content has this edge case AND the normal creation and usage differ.

  One way to fix this is to calculate and interpolate the normal, bi-tangent, and tangent at the pixel level where you need 4 partial derivatives. Unfortunately that solution has a huge run-time performance penalty since now all normal mapped objects need to use this slower, correct method. It MAY be possible to detect this offline? That is probably worthy of a PhD paper! This would allow for the "faster, incorrect" method to be used 99% of the time, and the "slower, correct" way for the 1% of time where it matters.

  You are probably asking: Why doesn't everyone just pick one method and use that then?

  Actually, we did: :-)

• A lot of people used Eric Lengyel's original implementation. Unfortunately, at the time, many people didn't know it has problematic edge cases that would fail.

  As the game development community investigated the problem it has slowly migrated to a new solution:

• Mikktspace has become the "de facto" standard in the game content creation. Additionally the "final normal" calculation is done at the per pixel level with a higher precision for interpolation – what is called a per-fragment TBN calculation.

  My understanding is that:

• Blender already uses mikktspace.

• Unity 5.3+ (IIRC) uses mikktspace.

• Unreal Engine 4 uses a combination of mikktspace and per-fragment.

• Other exporters may give you an option for mikktspace.

  The mikktspace and per-fragment TBN calculations are the two solutions I've been looking at and seeing how to best integrate it looking at the trade offs involved.

  This is probably far more details then you wanted but hopefully you found my attempt trying to summarize a rather complicated topic helpful and it shed some light on how the problem came about, the current implementation, and possible future solutions.

[sl-service-account]

Kyrah Abattoir commented at 2021-05-24T21:46:59Z

No that is quite fine, it brings up interesting issues related to normal orientations that I was not aware of, and in depth explanations are always welcome! Thank you for spending the time explaining it!

[sl-service-account]

nergal.ninetails commented at 2021-06-08T22:57:30Z, updated at 2021-06-08T23:41:59Z

The problem is likely closely related to this BUG-227751 There are ways around this issue currently, but the workarounds are inefficient for performance, and just overall a pain to do. The more Items I make, the more I find that it happens more than just if the item as angler UV's, but different poles West and East seems to be the biggest offenders.

One tip I can give is, ensure that if you have two seems in the UV space, ensure that they are going the same direction.

 

If you have an island @ 0° and another rotated @ 90° that share the same UV seem, this will create issues.