Lightmaps and metals

[rawnsley]

In MeshStandardMaterials lightmaps are applied to the diffuse light component, but not the specular reflections. This makes sense because shadows don't fall on mirrors, but I think the logic isn't quite right for rough metals. Obviously smooth metals are a special case, but rough metals such as brushed aluminium do have shadows.

Blender's Principled BSDR shader in Cycles renders shadows on metal:

The pods in the scene are arranged with roughness going from 0% to 100% bottom-to-top and metallic going from 0% to 100% left-to-right.

If I generate a lightmap for this scene in Blender and export that for use in three.js (via Mozilla Hubs using their GLTF extension) I get the following output:

The top row clearly shows the shadow effect diminishing with the increase of the metallic factor.

In simplistic terms, this could be fixed if the scattered light from the rough metal surface (the diffuse-like part of the specular component) was also attenuated by the lightmap.

[mrdoob]

Thanks for the example. The explanation makes sense to me 👍

[WestLangley]

... this could be fixed if the scattered light from the rough metal surface (the diffuse-like part of the specular component) was also attenuated by the lightmap.

Light maps in three.js do not attenuate light -- they are additive to total light. Light maps model non-directional, ambient light. See the discussion in #6263.

...I generate a lightmap for this scene in Blender and export that ...

Also see the discussion in #21912.

A live example would be helpful if this issue is to be pursued.

[rawnsley]

@WestLangley Here is a live example (adapted from the one by @netpro2k): https://rawnsley.github.io/simple-lightmap/ and here is the source including the Blender file: https://github.com/rawnsley/simple-lightmap

It demonstrates this issue well enough, but I can't work out why the contrast from the lightmap shadows is so much lower than in Cycles.

[netpro2k]

I think the contrast issue is seperate from the primary discussion here about metalness, but here is what I have found anyway. I think comes down to 2 things:

1. In Three the envmap is applying diffuse lighting again that is already "baked in" to the lightmap. In Hubs we avoid this by not applying this component to lightmapped objects:

THREE.ShaderChunk.lights_fragment_maps = THREE.ShaderChunk.lights_fragment_maps.replace(
  "iblIrradiance += getLightProbeIndirectIrradiance( geometry, maxMipLevel );",
  `#ifndef USE_LIGHTMAP
     iblIrradiance += getLightProbeIndirectIrradiance( geometry, maxMipLevel );
   #endif`
);

Unclear if this is actually a sane thing to be doing, but seems to make sense.

2. The tonemapping in Blender and threejs differ. "Filmic" in blender and "ACES Filmic" in three are not quite the same. Working on a fix for this in Hubs by exporting the tonemapping from Blender as a LUT (using OpenColorIO tooling) and then using that in three for tonemapping. This code snippet doesnt work without some changes to three to set the toneMappingLUT uniform, but here it is anyway:

   THREE.ShaderChunk.tonemapping_pars_fragment = THREE.ShaderChunk.tonemapping_pars_fragment.replace(
   "vec3 CustomToneMapping( vec3 color ) { return color; }",
   `
   precision highp sampler3D;
   uniform sampler3D tonemappingLUT;
   vec3 CustomToneMapping( vec3 color ) {
   // Match Blender's exposure curve
   // color *= pow(2.0, toneMappingExposure);
   
   float lutSize = 32.0;
   float pixelWidth = 1.0 / lutSize;
   float halfPixelWidth = 0.5 / lutSize;
   vec3 uvw = vec3( halfPixelWidth ) + color.rgb * ( 1.0 - pixelWidth );
   
   return texture( tonemappingLUT, uvw ).rgb;
   }
   `
   );

   This has things almost exactly the same for unlit materials, so then the rest can actually be a fair comparison of the lighting. Being able to provide a custom LUT for tonemapping also just seems generally useful for other usecases, so I will likely try and submit this as a PR upstream of our fork if there is interest.

[WestLangley]

@netpro2k Your code in (1) no longer exists, so you appear to be using an old revision of three.js. Be aware of that if you upgrade.

[WestLangley]

@rawnsley What is exported from blender as a "light map" is likely not what three.js considers to be a light map.

Also, for pure metals, the diffuse reflectance is zero, so a three.js light map will add nothing to total illumination in the case of a pure metal.

[netpro2k]

@netpro2k Your code in (1) no longer exists, so you appear to be using an old revision of three.js. Be aware of that if you upgrade.

Good to know. We plan to do another ThreeJS update in Hubs very soon, will need to migrate this assuming it still makes sense.

[mrdoob]

@rawnsley What is exported from blender as a "light map" is likely not what three.js considers to be a light map.

@rawnsley @netpro2k Do you know if Unity, Unreal, Godot, ... treat light maps the same way Blender does?

[rawnsley]

@rawnsley What is exported from blender as a "light map" is likely not what three.js considers to be a light map.

@rawnsley @netpro2k Do you know if Unity, Unreal, Godot, ... treat light maps the same way Blender does?

As there's no specific lightmap concept in Blender it's possible that the downstream tools have different expectations, but I'm afraid I don't know about any other engine except Filament, which doesn't support them explicitly at the moment.

This is how we are currently creating our lightmaps using Blender's generic baking system:

[mrdoob]

but I'm afraid I don't know about any other engine except Filament

Any change you could have a look at them? The more data you can gather the easier it'll be for us to work on this.

[WestLangley]

Light maps in three.js encode non-directional, ambient light. So baking direct light sources from Blender is not consistent with three.js.

[j-conrad]

Just wanted to add some more insight into this issue as I've been experimenting with Blender and exporting to Hubs.

My finding about Blender: Setting any material to have a Metallic value of 1.0 will cause a discrepancy between how Cycles renders shadows vs. how Cycles light bakes shadows. Is this a bug? Not sure yet. There may be a good reason for it, but so far I've found nothing.

Example

The test scene has a simple box with two small columns inside it to cast shadows. There are two identically-placed boxes inside the room-- one is Zero % Metallic and the other is 100% Metallic. Both are 50% Roughness. I have purposefully NOT applied any textures to the boxes. The light comes from an HDR sky and 1 white spotlight inside the box shining toward the columns which cast shadows on the boxes equally (symmetrically). In this image, you can clearly see the result of the Cycles lightbake on the left vs. the result of the Cycles render on the right. Anything 100% metallic seems to trigger the Cycles lightbake to completely darken or otherwise bypass its surfaces. The resulting lightmap is completely black where the UVs of those faces reside. TL;DR In this example, the lightmap shadows not showing in Hubs (threejs renderer) is a result of Blender never rendering them to begin with.

Now, if we set the Metallic box to instead be 99% metallic or below, its surfaces get calculated into the lightmap texture:

So, now that that's out of the way, we can just focus on what's happening in Hubs.

If you'd like to see it firsthand, Here is a link to the scene

Now in this example, I've rebaked the lightmap with the Metallic value set to 99%. So now there are indeed shadows on the lightmap where that metallic box's UVs are located. (see previous image)

The result in Hubs is the same. The renderer does not display lightmap textures on the metallic box. Screenshot from Hubs:

However, the interesting part is that Three seems to understand how to handle punctual lights and shadows on metallic objects just fine. (I'm not actually sure of the accuracy, but it appears to be correct visually anyway.) Here's that same scene in Hubs with a punctual dynamic light with shadows casting a shadow onto the 99% Metallic box:

Anyway, I hope this helps to make things even clearer.

[mrdoob]

Here's that same scene in Hubs with a punctual dynamic light with shadows casting a shadow onto the 99% Metallic box:

I think this is a good example.

To me it makes sense that the scene using a lightmap and the scene with a spotlight should look similar.

[j-conrad]

EDIT: I was mistaken about this. Please disregard

I have an update that might help...

In all my examples and testing, I was always using the Principled BDSF Metallic slider.

It turns out, that if you use a metallic image texture--even a uniform one with the same exact value(s) as the slider, the lightmaps render just fine!

For clarity, the above example graph yields the 'bad' results outlined in previous examples, but the following graph works:

[mrdoob]

@j-conrad you mean that a white image behaves differently than a slider at 1.0?

[j-conrad]

EDIT: Nope. I was incorrect.

Exactly. But the value itself is irrelevant. Any slider value alone doesn’t work. But an image texture with any value does work.

[mrdoob]

I see... Is that a bug on Blender's side?

[j-conrad]

EDIT: Doesn't seem to be a bug.

Unclear, but it’s likely a bug with the glTF Exporter which is maintained separately here: glTF importer/exporter

[julienduroure]

Not sure what can be wrong in slider export. Seems the value of slider is correctly exported:

Are you sure everything is correctly manage on your side when there is no metallicRoughnessTexture, but only metallicFactor and roughnessFactor?

[j-conrad]

Ugh... My sincere apologies about that last bunch of info regarding Slider values vs. Texture values. I seem to have mistaken some other variable involved in my particular scene setup.

I'm not even sure this would be considered a bug anymore, but I do think the discrepancy between how punctual lights and lightmaps behave on metallic surfaces is still worth looking into.

Here, I'm using a metallic texture that is set up in progressive values (by tens). The uppermost (100) and lowermost (0) areas are a little bit larger than the other stripes.) There is a lightmap texture on the surface of the plane. The baseColor value is medium grey.

You can see that the lightmap texture does indeed show on most of the plane, until you get into the 90%+ range where it drops off precipitously. I'm curious where that curve was decided, be it physically-based or otherwise.

But just like the earlier example, a punctual light casting a shadow shows up very differently across all metal values.