Shading issue

[Malgalad]

Hi guys! I'm having another trouble. I've included shadow mapping into my scene, but it doesn't work as I expected. Here's the simple demo: http://malgalad.dyndns.org/demo. As you can see, shadows are correctly projected on the Cylinder geometry, but shadows on the character... they're completely broken. I'm actually porting shaders from Aion: The tower of eternity (CryEngine 1), and this is how it should look ingame: http://ipic.su/img/img3/fs/Aion0003.1331749429.jpg - hair drops shadow on the face, shoulder pads - on the arms, face itself - on the neck etc. I don't have any idea where's the solution, do you have any ideas?

[alteredq]

That's probably because you are using alpha testing. For visual rendering some parts of triangles are thrown away but for shadows whole mesh is visible.

Solution is to use custom depth material matching the visible material.

mesh.customDepthMaterial = myDepthMaterial;

You can create custom depth material as ShaderMaterial by copying depthRGBA shader, adding there uniform with your texture and then using this to discard invisible fragments in the same way how you do it for the visible material.

https://github.com/mrdoob/three.js/blob/master/src/renderers/WebGLShaders.js#L1668

[Malgalad]

To be exact, I have to play around smth like that?

'depthRGBA': {
    uniforms: {
        'texture': {type: 't', value:0, texture: []}
    },
    vertexShader: [
        THREE.ShaderChunk[ "morphtarget_pars_vertex" ],
        "void main() {",
            "vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );",
            THREE.ShaderChunk[ "morphtarget_vertex" ],
            THREE.ShaderChunk[ "default_vertex" ],
        "}"
    ].join("\n"),
        fragmentShader: [
                                      "uniform sample2D texture;",
                                      "vec4 pack_depth( const in float depth ) {",
                "const vec4 bit_shift = vec4( 256.0 * 256.0 * 256.0, 256.0 * 256.0, 256.0, 1.0 );",
                "const vec4 bit_mask  = vec4( 0.0, 1.0 / 256.0, 1.0 / 256.0, 1.0 / 256.0 );",
                "vec4 res = fract( depth * bit_shift );",
                "res -= res.xxyz * bit_mask;",
                "return res;",
            "}",
                                      "void main() {",
                                                         "if (texture.a < 0.5)",
                                                                            "discard;",
                                                         "gl_FragData[ 0 ] = pack_depth( gl_FragCoord.z );",
            "}"
        ].join("\n")
    }

Am I right?

[Malgalad]

I guess, I was right :) I created that shader, but it only changed the shadow on the cylinder so that it now respects alpha, but still no changes for a character. I've updated demo, so you can see this. Here it is. With standard shader: http://ipic.su/img/img3/fs/before.1331756662.png With custom shader: http://ipic.su/img/img3/fs/after.1331756709.png

[alteredq]

You can try to play with shadowBias.

Your particular model + light direction setup seems to respond best to small negative bias. I got somehow okeish shadows with light.shadowBias = -0.0002:

Just seems you've got your material too bright, probably to compensate for self-shadowing, you need to tune ambient and directional lights; shadows look more "in place" when lighting is different:

Also make light frustum as tight as possible (in all six planes).

Plus as this model has a lot of open surfaces, you may want to try setting:

renderer.shadowMapCullFrontFaces = false;

This will get you different type of artefacts.

This is a complex model seen from very close, so I think there'll always be some artefacts.

You'll need to experiment and see what looks the best. Tweaking shadow maps is more art than science. If you play with numbers (light frustum, shadow bias, light direction, shadow map resolution), you'll get a feel for what are tradeoffs.

[Malgalad]

Oh, man, thank you! This just looks awesome! The main problem was that i had rather small shadowCameraNear, 0.1, setting it to 800 (light is on 1000 units above, to imitate direct light, and character is about 170 units in height) created this shadows.

But now I have a few more questions:

1. Even though it looks fine, on the face (and breasts with legs) you can see strange artifacts, like shadow map was computed for a flat-shaded model. Am I right in my suggestion, or is that such a type of artifact, that should always appear since it is "a complex model seen from very close"?
2. I guess shadow mapping doesn't respect normal mapping, right? It's not a big deal, but I think it should be great. Than I have to modify my custom depth shader somehow, yeah?
3. How to make shadow less dark? When I change shadowMapDarkness, I don't see any differences.

And a bit of offtopic - how to create on GitHub formatted text like in https://github.com/mrdoob/three.js/issues/1135? In "GitHub Flavored Markdown" I've only found ```javascript, but it seems that it doesn't work for me...

[alteredq]

That looks much better than before ;)

1. Even though it looks fine, on the face you can see strange artifacts, like shadow map was computed for a flat-shaded model
2. I guess shadow mapping doesn't respect normal mapping, right?

Shadow map is computed from the real geometry, so things that are just "painted" on surface have no effects (like smooth shading or normal maps), the only thing that enters shadow computation is mesh outline.

But these artefacts on the face are not caused by this. Most likely it's coming from light direction vs triangles orientation. Some combinations perform really badly (more parallel the worse). You can try to move your light in different directions, you may hit better or worse artefacts.

Also you can try to turn off shadow filtering:

renderer.shadowMapSoft = false;

This will give you different type of artefacts.

Than I have to modify my custom depth shader somehow, yeah?

Not really, this would be more complicated, you would need to somehow compute shadows from normal map separately in normal map shader and mix them with shadows from shadowmapping. I think this is done by raycasting using heightmap and in general is really slow.

1. How to make shadow less dark? When I change shadowMapDarkness, I don't see any differences.

It's light.shadowDarkness

And a bit of offtopic - how to create on GitHub formatted text like in https://github.com/mrdoob/three.js/issues/1135? In "GitHub Flavored Markdown" I've only found ```javascript, but it seems that it doesn't work for me...

I use javascript and glsl (not sure if this one has any effect, maybe it just falls back to something generic). Also make sure quotes have the right orientation.

[Malgalad]

OMG, I love you! Changing light direction was good enough.

But of course, some more questions :)

1. I saw physically based shading parameter in shadowmap chunk, what does it do? Also, what does shadowCameraTop, shadowCameraLeft etc. change?
2. Is there a way to make shadows more "soft", with more blurry edges?
3. When dynamically turning shadows off and on (via renderer.shadowMapAutoUpdate), if I turn shadows off the model renders like it is completely in shadow, it's too dark. So should I just increase intensity of other lights or is there any other way to deal with it?
4. Can I modify THREE.js library code by adding some custom code like this? THREE.ImageUtils.loadTexture = function(){}; I need to handle onerror event in this loader, and also some shaders fixes to make it working into Opera 12 alpha. I've made this changes into the minified library, but it look rather "dirty" for me.

[alteredq]

1. I saw physically based shading parameter in shadowmap chunk, what does it do?

Different specular term, more accurate but also more costly.

Also, what does shadowCameraTop, shadowCameraLeft etc. change?

Light frustum for directional lights (which use OrthographicCamera).

2 . Is there a way to make shadows more "soft", with more blurry edges?

Not really. You could try to use smaller resolution shadow map, but then it would just look uglier.

We would need something like this: http://www.glge.org/variance-shadow-maps-in-webgl/

Variance shadow maps however come with own specific artefacts, in some situations there are very ugly light patches inside shadowed areas. And they are more expensive. Still may be worth trying. One day.

3 . When dynamically turning shadows off and on (via renderer.shadowMapAutoUpdate), if I turn shadows off the model renders like it is completely in shadow, it's too dark. So should I just increase intensity of other lights or is there any other way to deal with it?

You don't turn shadows on/off with renderer.shadowMapAutoUpdate. It does exactly what it says, it just stops updating shadow map (which means it "freezes" shadow, if it was in some strange undefined state, it'll stay like this).

If you want to turn off shadows dynamically, you will need to turn off shadow casting on all objects that cast shadow. Shadow receiving can't be turned off, this is baked in materials, but you can turn off shadow casting to achieve shadow-less look.

4 . Can I modify THREE.js library code by adding some custom code like this?

Whatever works for you ;). That's beauty of git, everybody works on own clone and if there some piece of code interesting for others, they can merge it. Fixes for Opera would be interesting.

[Malgalad]

1. Thx, understood. Since I'm using spot light, I don't need it.
2. OK. Changing shadow map resolution is fine, since ingame they don't have high resolution either.
3. Great. That's exactly what I wanted to achieve. Solution with shadowMapAutoUpdate I found in one of the closed issues...
4. I meant, will functions, that replace default, work correctly? But I guess, yes. Anyway, I have to try. About Opera - it's not "fix" as you might expected, I don't know, if there's in English slang equivalent to russian "spike-nail", that means that it's a temporary solution made to just have code working. Now Opera doesn't have all() function in fragment shader, so I had to replace each inclusion with if statement. That's not a THREE.js's, but Opera's problem.

Oh, I have another question. Now I've encountered a problem with half-opaque textures like this:

Those textures doesn't drop shadows cause they have opacity less then alphaTest. So, I think, I have to add new parameter, that modifies shadowDarkness. I know how to do it, but I don't know how to transfer it to the actual shader. Depth textures uses vec4, but is it necessary, can't I use one of the channels for this new parameter? To be exact, how do I modify pack/unpack functions? Or it's not possible at all?

[alteredq]

Now Opera doesn't have all() function in fragment shader, so I had to replace each inclusion with if statement. That's not a THREE.js's, but Opera's problem.

Bah :S. I had to switch from ifs to all because this was not working on ATIs.

Though this was with if( a && b && c ) syntax, maybe nested ifs would work. Just this would be really ugly and probably bad for performance, branching in shaders is no-no, the less of it the better.

Those textures doesn't drop shadows cause they have opacity less then alphaTest.

I don't know how to do shadows for semi-transparent things. I didn't even start touching that topic. Regular shadowmaps are just on/off, either you have shadow or you don't, there is no in between.

Transparent object alone are massive pain, I expect transparent objects + shadows would be even worse.

[Malgalad]

I don't know how to do shadows for semi-transparent things. I didn't even start touching that topic. Regular shadowmaps are just on/off, either you have shadow or you don't, there is no in between.

I have an idea. Now THREE.js packs depth float into rgba, right? What if we will pack depth into rgb and use alpha-channel for shadow darkness? Yes, we will have less precision, so it might not be such cost-effective on meshes with no semi-transparent faces as on meshes with 'em... But, i guess, it's the only solution. I'll try it and post the results (now I've already try using 3 bytes instead of 4 and saw no visual differences, but didn't implement shadow darkness yet).

Transparent object alone are massive pain, I expect transparent objects + shadows would be even worse.

IMO it can be solved manually by adding callback function and flags. E.g. with this veil (see the lat pic.) - I'll add special filed, like 'add this mesh only after loading other meshes', and flags to other meshes if they've loaded and rendered or not. And when they have rendered - render this mesh. It's a bit complicated, and probably wont work for intersecting meshes, but it's better then nothing.

UPD. Looks like, I've made it :) Old shaders:

New shaders:

However, there's a bug: if a shadow from semi-transparent object drops over any other shadow, this shadows do not sum. Like here:

I don't know the solution yet, except smth like local raytracing to check if there's more than one shadow...

[Malgalad]

In the last Opera build (12.00.1359) all() function was added (CORE-44337). So, it's working 'from the box' now :)