[MiniEngine] No light in glTF mode

[MacGeorges]

Hi

My DirectX knowledge is still low and all these samples are an amazing resource. I specifically enjoy the MiniEngine and like to play around with it.

There is still one thing I can't grasp: In Legacy mode, the MiniEngine loads a h3d version of the Sponza scene, and everything works well out of the box.

However, if I disable the Legacy mode, the glTF version is loaded (or any glTF file I want), but no light is applied. The sun intensity / orientation / etc. settings have no effect, nor the lights I create through code.

Am I missing something or is there a bug here?

MacGeorges

[stanard]

Oh, thanks for the nice words. I hope there isn't a bug, but the glTF renderer uses a different set of shaders and relies on image-based lighting (IBL) for the shading model. There is a collection of cube maps in ModelViewer/Textures that can illuminate the model. Are you loading sample glTF models or some of your own? I tried to convert Sponza to glTF but I couldn't make it look very good with a single environment map. (It really needs multiple light probes or an indirect lighting solution.)

[MacGeorges]

The nice words are sincere. Yes, the dds files are loaded. Will look into the shaders then, thank you for the tip. The model I am using is the provided Sponza/pbr/sponza2.gltf, but results are the same with the recommended samples.

[stanard]

I loaded the Sponza glTF 2.0 model and think I can explain what you're seeing. There is no direct lighting applied and therefore no direct shadowing. Diffuse and specular environment maps are applied without direction (except the surface normal is taken into account) so you can't have a directly shadow. What is typically done to apply shadow maps is the direct sunlight component is removed from the environment map, leaving only indirect lighting such as the sky emission, and then the sun light is applied in a traditional manner.

[MacGeorges]

Thank you for enlightening me (pun intended).

My current "fun goal" is to get the same rendering as the "legacy" mode (h3d file), but with the glTF.

[stanard]

That's a worthy challenge. Be aware that the textures are modified to support the physically based rendering model. There are terms for "metal" and "roughness" as well as the typical normal and albedo textures. There would also be an AO term for pre-baked ambient occlusion, but I don't think these textures have it (and we have screenspace AO).

If I were you, I'd copy the code for sunlight and shadow from the legacy renderer and see what it looks like.

[YunmaoLeo]

For those who are seeking for a possible solution of adding direction light shadows of glTF renderer, here is my approach:

1. Preapre Shadow Buffer: Function -- ModelViewer::RenderScene() is where mini-engine draw each frame. In RenderScene(), we have a scope "Sun Shadow Map", it is designed to fill g_ShadowBuffer in. Similar to the approach of filling depth buffer, in Sun Shadow Map, we invoke RenderMeshes() with camera pos of _mCamera and viewProjMatrix of m_SunShadow(where the light is). Note, we need to overload function Sorter::RenderMeshes in order to specify a viewProjMatrix before render.

           {
               ScopedTimer _prof(L"Sun Shadow Map", gfxContext);
   
               MeshSorter shadowSorter(MeshSorter::kShadows);
               shadowSorter.SetCamera(m_Camera);
               shadowSorter.SetDepthStencilTarget(g_ShadowBuffer);
   
               m_ModelInst.Render(shadowSorter);
   
               shadowSorter.Sort();
   
               shadowSorter.RenderMeshes(MeshSorter::kZPass, gfxContext, globals,m_SunShadow.GetViewProjMatrix());
           }

2. Handle Shadows in Shader: In default, this renderer use DefaultPS.hlsl as pixel shader. Fortunately, a piece of code computing shadow is already there, which surrounded by #if 1 (disabled by default). By simply active these code, we get a rough shadow with aliasing.

3. Add Simple Percentage Closer FIltering (PCF): In origin DefaultPS.hlsl, sunShadow is compute by single sampler, we apply a PCF here.

   float shadowTexelSizeX = 1.0f/2048.0f; //2048 == shadowBuffer.Height()
   const float Dilation = 2.0;
   float d1 = Dilation * shadowTexelSizeX * 0.125;
   float d2 = Dilation * shadowTexelSizeX * 0.875;
   float d3 = Dilation * shadowTexelSizeX * 0.625;
   float d4 = Dilation * shadowTexelSizeX * 0.375;
   float sunShadow = (
       2.0 * texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy, ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2(-d2,  d1), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2(-d1, -d2), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2( d2, -d1), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2( d1,  d2), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2(-d4,  d3), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2(-d3, -d4), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2( d4, -d3), ShadowCoord.z ) +
       texShadow.SampleCmpLevelZero( shadowSampler, ShadowCoord.xy + float2( d3,  d4), ShadowCoord.z )
       ) / 10.0;
   
   colorAccum += ShadeDirectionalLight(Surface, SunDirection, sunShadow * SunIntensity);
   
   uint2 pixelPos = uint2(vsOutput.position.xy);
   float ssao = texSSAO[pixelPos];
   
   Surface.c_diff *= ssao;
   Surface.c_spec *= ssao;
   
   // Old-school ambient light
   colorAccum += Surface.c_diff * 0.1;

Then we get a better anti-alias result.