AO should only modulate ambient, not direct lighting

[echadwick-artist]

AO seems to be darkening direct lighting, but as far as I understand it AO should only darken ambient lighting.

Here's an example exported from 3ds Max 2018.4, using Max2Babylon 1.2.24, and rendered in http://sandbox.babylonjs.com/.

The torus knot model has white diffuse, black metalness, white roughness, and baked ambient occlusion.

The sphere models have white diffuse, black metalness, white roughness, and white/gray/black ambient occlusion.

There is a single light, from the upper left front.

Black sphere should have zero ambient lighting because the AO texture is black, but it should still be lit by the direct light because the color texture is white.

Any clues why it is doing this? Thanks for your help!

AmbientOcclusion_TorusKnot.zip

[sebavan]

Hello,

we are considering every diffuse lighting being subject to AO so both diffuse and environment irradiance (probably what you are calling ambient ???) is subject to ao:

The shader fromula being: finalColor = finalDiffuse ambientOcclusionColor + finalIrradiance ambientOcclusionColor +...

The AO is basically masking all the diffuse lights.

If this is the wrong way of doing it, do not hesitate to forward the references you are using so that we could check how to integrate it maybe as a separate mode to not break back compat ?

[echadwick-artist]

Some info here and here https://marmoset.co/posts/physically-based-rendering-and-you-can-too/#ao https://academy.allegorithmic.com/courses/05171e19aa8dc19421385fd9bb9e016e

[keyoss]

Part of it is correct.

AO is generated for Ambient light and diffuse light, you can spot this if zou have a close look in the corners of your white walls. There are dozens of technical papers out that will exactly explain how AO works, physically.

While we are in the "Virtual/digital Environment/World" it would happen, just for the artistic or designers look that IF you use a spotlight or flashlight that the corner where AO happens will be lit. Physically this will not happen since the AO will just get thinner. In our digital world we WANT to be able to overwrite the AO for dark corners with a direct- flash- or spotlight.

So the difference, which you can read from the marmoset guys is: we are using diffuse and diffuse lighting channels. same goes for specular: we have specular and specular lighting channels. and the same for Ambient.....

So basically all of it is partially correct.

i can show you an example how our material system works:

We used and build the material system in conjunction with marmoset, substance and LYS from knald to form a more or less industrial standard (which was not defined 3-5years ago for PBR and IBL)

[sebavan]

This is really interesting and I am wondering if we should have a parameter to chose the amount of occlusion applied to diffuse punctual lights.

[sebavan]

@keyoss could you define in your schema what diffuse light vs specular light vs ambient light means ?

In Babylon we have punctual lights (spot, point....) which generate both diffuse and specular

We also have environment light (IBL) which generates both diffuse (irradiance via sh) and specular (reflection)

Relying on PBR ambient is usually mixed in the SH.

So just to be sure, your proposal would be to chose the amount of AO we apply to the diffuse part of the direct lighting ?

[echadwick-artist]

I'd like to see all ambient (diffuse IBL, HemisphericLight, ambientColor) modulated by the full ambient occlusion texture. But I'd prefer the diffuse and specular from Point/Directional/Spot lights and the specular from IBL to only be affected weakly by the bottom 10% or so of the ambient occlusion map (or else not at all).

Edit: in my example, the middle sphere's grey AO should show the full intensity lighting from the point light, but only half of the IBL. The bottom sphere's black AO should be fully occluding the IBL, and only partially occluding the point light.

I'm not sure how much a black AO should partially occlude direct lighting, but just think of the flashlight example as a guideline... if the underside of a car has black AO and bend down to shine a flashlight on it, what would I see? It would still occlude the ambient lighting, but the flashlight would illuminate it quite well.

[sebavan]

OK so I will not go as far as Hemispheric light (I ll still consider it as analytical light)

I won t dim the specular at all cause it is a rather big changes and could confuse a bit the community.

I can add a value like ambientImpactOnDiffuse in the material that would let you decide how much the ambient is impacting the diffuse part of the analytical lights. By default it would be one to not introduce any back compat issues and people could decide to put the value to 0 to completely remove the impact of the ao map on analytical lights.

Would that work for you ? it would not break back compat and should let you decide how much impact AO is having on the diffuse lights.

Rather than modifying hemlights + ambient color, I would advise you to bake them in the sh used for irradiance so that they would be subject to AO as it should.

@e-chadwick would that work for you ? if yes I ll integrate it first thing on Monday.

[echadwick-artist]

I understand the desire to not cause backwards compatibility issues, but what if the existing method is completely wrong? Wouldn't content creators want their existing content to be rendered correctly? Just my 2 cents though, up to you guys.

Otherwise, sounds great.

What kind of support would I have in 3ds Max to control the output of GLBs via Max2Babylon? I can't edit those after export, unless I'm mistaken, so I would need to edit the ambientImpactOnDiffuse in 3ds Max somewhere. Maybe a setting in the exporter window, similar to how the Merge AO Map setting is exposed there? I think it should default to a low value, like 0.1, for better default physical accuracy.

[sebavan]

Oh It is related to gltf, let me check with the gltf guyz what is planned for AO and Punctual Lights as we should follow the spec on this part.

[echadwick-artist]

I'm curious about your reasoning behind HemisphericLight and AO.

It's considered an ambient light in the docs. It behaves the same as other ambient sources because it's a very low-detail light source, emulating sky dome and ground bounce. So then shouldn't it be occluded by AO?

As a content creator, I would treat this as a simplified ambient light, applying it as a lighter-weight replacement for IBL. I would expect it to be occluded by AO the same way that diffuse IBL should be. But maybe I'm not understanding the other use cases.

[sebavan]

I agree with you but as with PBR material we always use IBL I prefer to push towards the integration of those values in the harmonics preventing the computation of another light in the shader.

I like to think we ca use almost exclusively use IBL for ambient. It also simplifies a lot the introduction of the AO behaviour which can be applied only during the final composition in the shader :-)

[sebavan]

I am currently on discussion with some ppl from the gltf side of things to see their point of view on this and I will apply the chosen method in BabylonJS for the loader.

There are strong chances it ends up being as you described.

[echadwick-artist]

Thanks for looking into this sebavan!

[sebavan]

I should have an answer early next week as everybody is currently at Siggraph discussing WebGL and GLTF :-)

[sebavan]

@echadwick-wayfair the gltf discussion might take a bit longer to ensure it fits with all major browsers and does not disrupt the way the community is working.

I will then add a parameter that you could globally control in Babylon so that it unblocks you.

Then once the choice will be made for the spec, it would be all ready in the engine to be used by the loader.

ETA is Monday for your parameter.

[echadwick-artist]

Thanks sebavan, great news. Fairly minor appearance-wise to some of my cohort, but speaking as a visual artist it makes a pronounced difference to me!

[sebavan]

@echadwick-wayfair with the latest nightly, you can now define on a per material basis the impact of AO on analytical lights:

Or you can define the default value for your app by overriding the default value on the pbr material: BABYLON.PBRMaterial.DEFAULT_AO_ON_ANALYTICAL_LIGHTS = 0;

You should run this code before the creation of your pbr materials or before loading your assets as they will create pbr materials.

I am closing the issue for now and we ll take care to follow the gltf spec once defined in the loader.