erich666
commented
3 months ago To give a visual sense of where there are bad normals, here's a view of the "heatmaps" produced by my analysis program. Bright green means the texel has a normal length that's pretty far from 1.0, dark green means close but not quite right. This view shows the problems for all textures found to have errors. As explained above, some of these I have not submitted for update as the problems didn't seem significant enough to me, such as those with the Flight Helmet.
emackey
elalish
Some normals textures are poorly formed, with too-long or too-short normals and other problems. This commit addresses the worst PNGs. It does not fix the JPEG versions a few have (see notes below), KTX files, nor makes new GLB files for the corresponding glTF-Binary directories. See https://github.com/KhronosGroup/glTF/issues/2368 for background, etc.
Normals analyzed and cleaned by my new project https://github.com/erich666/NormalTextureProcessor. My notes are below, full analysis report from the program is attached in analysis.txt. analysis.txt
A better fix for the models with problems, in my opinion, would be for the creators to properly form their normals textures from their original heightfield textures. Barring that, this submission at least attempts to clean up these non-compliant files. The main problems found were normals that were not near 1.0 in length, for various reasons (e.g., for some, the blue - Z - channel was never used at all, being 255 for all texels).
Avocado_normal.png - The blue channel values in file 'Avocado_normal.png' all have the same value: 255. That's usually a sign Z is not saved at all. Bad normals, with the Z values (assuming range -1 to 1) found to be as far off as 53 levels in expected value. Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.28911 (these should be near 1.0).
BarramundiFish_normal.png - Reason: the percentage of unnormalized XYZ vectors (assuming Z -1 to 1) is 18.0848% (758533 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.41678 (these should be near 1.0).
BoomBoxWithAxes_normal.png - 96.4118% of the XYZ normals are normalized well (4043804 texels), 0.304532% are close (12773 texels), and 3.28367% are pretty bad (137727 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.71857 (these should be near 1.0).
BoomBox_normal.png - 96.4118% of the XYZ normals are normalized well (4043804 texels), 0.304532% are close (12773 texels), and 3.28367% are pretty bad (137727 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.71857 (these should be near 1.0).
chair_fabric_normal.png - The blue channel values in file 'chair_fabric_normal.png' all have the same value: 255. This leads to long normals: Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.17733 (these should be near 1.0).
clearcoat_normal.png - short normals. Problem: bad normals, with the Z values (assuming range -1 to 1) found to be as far off as 85 levels in expected value. 46.2448% of the XYZ normals are normalized well (121228 texels), 6.31104% are close (16544 texels), and 47.4442% are pretty bad (124372 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.348711 and maximum 1.00223 (these should be near 1.0).
CopperPot_normal.png - short normals. Normal lengths found in the file, assuming Z -1 to 1: minimum 0.729064 and maximum 1.00315 (these should be near 1.0). 1.8713% of the texels' normal lengths are pretty bad (78488 texels)
Corset_normal.png - long normals. 69.0503% of the XYZ normals are normalized well (2896178 texels), 3.70004% are close (155191 texels), and 27.2497% are pretty bad (1142935 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.44507 (these should be near 1.0).
IridescenceAbalone_NormalBump.png - normals both too long and too short, almost half are out of bounds. 41.7248% of the XYZ normals are normalized well (875032 texels), 8.39934% are close (176147 texels), and 49.8759% are pretty bad (1045973 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.495346 and maximum 1.41665 (these should be near 1.0).
Lantern_normal.png - long normals. 86.4199% of the XYZ normals are normalized well (3624713 texels), 1.98977% are close (83457 texels), and 11.5903% are pretty bad (486134 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.41778 (these should be near 1.0).
material0_normal.png - bad normals, with the Z values (assuming range -1 to 1) found to be as far off as 249 levels in expected value. 84.031% of the XYZ normals are normalized well (3524515 texels), 0.560665% are close (23516 texels), and 15.4083% are pretty bad (646273 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.00679236 and maximum 1.0154 (these should be near 1.0).
SciFiHelmet_Normal.png - very short and very long normals, more than half of them are bad. 37.391% of the XYZ normals are normalized well (1568292 texels), 8.06487% are close (338265 texels), and 54.5441% are pretty bad (2287747 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.0590844 and maximum 1.21795 (these should be near 1.0).
SpecularSilkPouf_normal.png - almost 4/5ths of the normals are bad lengths. 14.5627% of the XYZ normals are normalized well (152701 texels), 5.82752% are close (61106 texels), and 79.6098% are pretty bad (834769 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.628272 and maximum 1.10899 (these should be near 1.0).
StainedGlassLamp_base_normal.png - a small percentage of normals are very short. Given that the StainedGlassLamp files are supposed to be physically based and correct, the normals should all be stored correctly, so I submitted fixed versions. It's a question whether the original authors could provide better data. 95.7856% of the XYZ normals are normalized well (4017540 texels), 0.720286% are close (30211 texels), and 3.4941% are pretty bad (146553 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.376695 and maximum 1.02126 (these should be near 1.0).
StainedGlassLamp_glass_normal.png - short normals for more than a fifth of the texels. 73.0098% of the XYZ normals are normalized well (1531126 texels), 5.02076% are close (105293 texels), and 21.9695% are pretty bad (460733 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.647083 and maximum 1.00339 (these should be near 1.0).
StainedGlassLamp_grill_normal.png - The blue channel values in file 'chair_fabric_normal.png' all have the same value: 255. This results in long normals for a quarter of the texels. 71.3795% of the XYZ normals are normalized well (2993875 texels), 4.51286% are close (189283 texels), and 24.1076% are pretty bad (1011146 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.73205 (these should be near 1.0).
StainedGlassLamp_hardware_normal.png - short normals for about a tenth of the texels. 91.7091% of the XYZ normals are normalized well (3846559 texels), 0.786829% are close (33002 texels), and 7.50406% are pretty bad (314743 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.36844 and maximum 1.02096 (these should be near 1.0).
technicalFabricSmall_normal_256.png - short normals for nearly half the texels. 51.7609% of the XYZ normals are normalized well (33922 texels), 5.78308% are close (3790 texels), and 42.4561% are pretty bad (27824 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.658111 and maximum 1.00425 (these should be near 1.0).
TestMap_Normal.png - this was analyzed as a texture where the Z's were assumed to go from -1 to 1. However, it's actually mapping 0 to 1, see https://github.com/KhronosGroup/glTF-Sample-Models/issues/364. Which was erroneously closed, see https://github.com/KhronosGroup/glTF-Sample-Assets/pull/112#issuecomment-2008090112 - I would hope that thread will properly fix it, but just in case, I've included a newly-fixed version here.
ToyCar_normal.png - only a small percentage of normals are bad, but also, only a few normals are actually modified. Of these, many show up as incorrect. 99.5039% of the XYZ normals are normalized well (1043374 texels), 0.00867844% are close (91 texels), and 0.487423% are pretty bad (5111 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.934445 and maximum 1.00603 (these should be near 1.0).
WaterBottle_normal.png - The blue channel values in file 'chair_fabric_normal.png' all have the same value: 255. This means Z is likely not set at all. There are "only" 0.800657% pixels that are pretty bad normal length, but that's actually a lot: 33582 texels. Normal lengths found in the file, assuming Z -1 to 1: minimum 1.00002 and maximum 1.24513 (these should be near 1.0).
wicker_normal.png - some normals are quite short, some are quite long. 92.617% of the XYZ normals are normalized well (242790 texels), 1.21346% are close (3181 texels), and 6.16951% are pretty bad (16173 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.240306 and maximum 1.11203 (these should be near 1.0).
I did not submit fixes for the .GLB files that go with some of these models, nor KTX. Either let me know of how to create these GLB and KTX files properly, or create them yourselves. Files involved: \Github\glTF-Sample-Assets\Models\Avocado\glTF-Binary \Github\glTF-Sample-Assets\Models\BarramundiFish\glTF-Binary \Github\glTF-Sample-Assets\Models\BoomBox\glTF-Binary \Github\glTF-Sample-Assets\Models\ClearcoatWicker\glTF-Binary \Github\glTF-Sample-Assets\Models\Corset\glTF-Binary \Github\glTF-Sample-Assets\Models\IridescenceAbalone\glTF-Binary \Github\glTF-Sample-Assets\Models\Lantern\glTF-Binary \Github\glTF-Sample-Assets\Models\LightsPunctualLamp\glTF-Binary \Github\glTF-Sample-Assets\Models\PotOfCoals\glTF-Binary \Github\glTF-Sample-Assets\Models\SheenChair\glTF-Binary \Github\glTF-Sample-Assets\Models\SpecularSilkPouf\glTF-Binary \Github\glTF-Sample-Assets\Models\StainedGlassLamp\glTF-KTX-BasisU - note the KTX \Github\glTF-Sample-Assets\Models\TextureTransformMultiTest\glTF-Binary \Github\glTF-Sample-Assets\Models\ToyCar\glTF-Binary \Github\glTF-Sample-Assets\Models\WaterBottle\glTF-Binary
There are other PNG textures that are questionable, a bit wrong, but I decided not to submit fixes for (I can submit these if desired). Here's my summary analysis of each:
AnisotropyBarnLamp_normalbump.png - small number of bad pixels: the percentage of XYZ's that do not form normalized vectors for Z -1 to 1 is 0.56355% (23637 texels)
CarbonFibre_normal.png - Borderline accept. Looking at the "diff" image, the differences looked minimal overall, a few bad normals that were a bit short: 97.5285% of the XYZ normals are normalized well (255665 texels), 1.19286% are close (3127 texels), and 1.27869% are pretty bad (3352 texels). Normal lengths found in the file, assuming Z -1 to 1: minimum 0.956115 and maximum 1.00584 (these should be near 1.0). Note there's also a .GLB: \Github\glTF-Sample-Assets\Models\CarbonFibre\glTF-Binary
FlightHelmet_Materials_GlassPlasticMat_Normal.png - some Z values were found to be negative, 0.00174046% (73 texels). The lowest Z value found was -0.0980392
FlightHelmet_Materials_LeatherPartsMat_Normal.png - The lowest Z value found was -0.2. That's bad, but it's just a few pixels. Some Z values were found to be negative, 0.00221729% (93 texels).
FlightHelmet_Materials_MetalPartsMat_Normal.png - The lowest Z value found was -0.262745. Some Z values were found to be negative, 0.00903606% (379 texels).
FlightHelmet_Materials_RubberWoodMat_Normal.png - The lowest Z value found was -0.411765. Some Z values were found to be negative, 0.026989% (1132 texels).
MandarinOrange_Normal.png - For XYZ normal length, just 0.000572205% are pretty bad (6 texels). some Z values were found to be negative, 0.0787735% (826 texels). The lowest Z value found was -1
teacup_normal.png - only a tiny percent of the normals are not normalized well: 0.018692% are pretty bad (196 texels).
teasaucer_normal.png - only a tiny percent of the normals are not normalized well: 0.0679016% are pretty bad (356 texels).
There are also a number of JPEGs used to hold normals textures, search "_normal.jpg". These are sometimes quite off, but I did not correct these, as JPEG is a lossy format and so will by its nature show problems. These JPEG _normal.png files are in: \Github\glTF-Sample-Assets\Models\ABeautifulGame\glTF \Github\glTF-Sample-Models\Models\AlphaBlendModeTest \Github\glTF-Sample-Assets\Models\ChairDamaskPurplegold\glTF \Github\glTF-Sample-Assets\Models\ClearCoatTest\glTF \Github\glTF-Sample-Models\Models\DamagedHelmet \Github\glTF-Sample-Assets\Models\GlassVaseFlowers\glTF \Github\glTF-Sample-Assets\Models\PotOfCoals\glTF \Github\glTF-Sample-Assets\Models\StainedGlassLamp\glTF-JPG-PNG
Analyzing these, the Z values (assuming range -1 to 1) were found to be as far off as from 12 (chair_wood_normal.jpg) to 127 (StainedGlassLamp_grill_normal.jpg) levels in expected value.
List of JPEGs: bishop_black_normal.jpg bishop_white_normal.jpg Castle_normal.jpg chair_damask_normal.jpg chair_wood_normal.jpg chessboard_normal.jpg Default_normal.jpg HotCoals_normal.jpg King_black_normal.jpg King_white_normal.jpg Knight_normal.jpg MatBed_normal.jpg Pawn_normal.jpg Queen_black_normal.jpg Queen_white_normal.jpg StainedGlassLamp_base_normal.jpg StainedGlassLamp_glass_normal.jpg StainedGlassLamp_grill_normal.jpg StainedGlassLamp_hardware_normal.jpg
File by file analysis: all these JPEGs are not very good at representing normalized normals textures: bishop_black_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 23.1521% example: texel x:658, y:813 has an unfixed value of 121,117,237, giving XYZ -0.051, -0.082, 0.86, which gives a normal of length 0.865, much lower than 1.0 the "r" value should be 254, not 237. bishop_white_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 19.0579% Castle_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 3.68702% chair_damask_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 18.6443% chair_wood_normal.jpg - 6.54449% of the texel Z values (17156 texels) are more than two from being properly normalized. chessboard_normal.jpg - The Z values (assuming range -1 to 1) were found to be as far off as 13 levels in expected value. Default_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 14.2765% HotCoals_normal.jpg - the Z values (assuming range -1 to 1) were found to be as far off as 87 levels in expected value. King_black_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 10.2787% King_white_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 13.8481% Knight_normal.jpg - The Z values (assuming range -1 to 1) were found to be as far off as 61 levels in expected value. MatBed_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 17.7353% Pawn_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 10.3454% Queen_black_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 11.3538% Queen_white_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 9.84666% StainedGlassLamp_base_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 6.63347% StainedGlassLamp_glass_normal.jpg - the percentage of stored Z 0 to 1 values that do not make normalized XYZ vectors is 28.3462% (note range 0 to 1 was closer than -1 to 1) StainedGlassLamp_grill_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 20.7516% StainedGlassLamp_hardware_normal.jpg - the percentage of stored Z -1 to 1 values that do not make normalized XYZ vectors is 17.0238%
Also note that there are related binary glTF files for some of these JPEGs: \Github\glTF-Sample-Assets\Models\AlphaBlendModeTest\glTF-Binary \Github\glTF-Sample-Assets\Models\ChairDamaskPurplegold\glTF \Github\glTF-Sample-Assets\Models\DamagedHelmet\glTF-Binary \Github\glTF-Sample-Assets\Models\PotOfCoals\glTF-Binary \Github\glTF-Sample-Assets\Models\StainedGlassLamp\glTF-JPG-PNG