Improved Soft Shadow Mapping

[zz85]

I always find myself reading up on interesting topics and then finding it way challenging for a noob. This time, its shadow mapping and so I'll try summoning expert ninja @alteredq for help on this. What we have currently for shadowing mapping is

1. hard shadow mapping - which is the simplest form of shadowing mapping
2. percentage-closer filtering (PCF) - a implementation of soft shadows by sampling a kernel (right now 3x3 which is 9 point)

I'm attempting/proposing 3 changes that would give better shadows

1. linear depth map - this helps to reduce shadow acne, due to improved precision
2. exponential shadowing mapping (ESM) - this does a box blur on the shadow map so that shadows are softer. i've read that the results might be good enough that SSAO is unnecessary.
3. percentage-closer soft shadows (PCSS) - uses a variable penumbra that give good performance with realistic soft shadows.

Starting off with change no. 1 - linear depth map, it currently seems like I would need the far/near of light to remapped the linear depth map? Currently I can map write linear values to the depth map, but it seems like I cannot pull it out from the shadow map :( See https://github.com/zz85/three.js/blob/d55bef9406a88d2195bdaec5e85e49bebc8b847a/src/renderers/WebGLShaders.js#L1122

note to @mrdoob avoiding merging this branch. contains experimental/dangerous/fail stuff.

references: Shadow mapping in general - http://www.nutty.ca/?page_id=352&link=shadow_map#tabs-4 Integrating PCSS (with code) - http://developer.download.nvidia.com/whitepapers/2008/PCSS_Integration.pdf Nvidia paper on PCSS - http://developer.download.nvidia.com/shaderlibrary/docs/shadow_PCSS.pdf GLSL implementation of VSM, ESM, PCSS - http://code.google.com/p/dretchstorm/source/browse/base/glsl/deferredLighting_fp.glsl GDC presentation of soft shadowing mapping techniques - http://developer.download.nvidia.com/presentations/2008/GDC/GDC08_SoftShadowMapping.pdf

possibly related issues: #1927

[alteredq]

You are trying to do too many things at once. Go slowly and be careful with what you focus on. Even our current simple shadows took ages to get working ;)

Go as basic as you can, if some method is "exotic", there is probably a good reason for that (messy / difficult implementation, weird corner cases, depends on some HW features, doesn't work well for some common use cases, too slow, consumes too much bandwidth, etc).

I'll just repeat what I said in other thread - if you want to improve our shadow maps somehow, try variance shadow maps first.

Though I think you should check GLGE implementation, the one that you linked to produces weird looking results.

What we have currently for shadowing mapping is

Also we do have cascading shadow maps for directional light.

1. linear depth map - this helps to reduce shadow acne, due to improved precision

For some unknown reason we actually never had trouble with acne, instead we get different artefacts - Peter Panning and sometimes self-shadowing.

BTW I tried linear depth at the very beginning and I couldn't get it working either. You can still see the commented out code in the shader:

https://github.com/mrdoob/three.js/blob/master/src/renderers/WebGLShaders.js#L1947

[zz85]

i guess i'm trying too many things at once, although i always create a new branch off for each attempt, rather than running everything in a single codebase. i rotated through the different approach hoping to find some similarities and understanding.

if VSM is your recommendation i would give it a shot ;) i thought that ESM is a superior method over VSM. (VSM requires to store the depth and depth squared). Taking a quick look at GLGE's implementation, the depth in encoded in R8G8 and depth squared in B8A8 (16 bits each). https://github.com/supereggbert/GLGE/commits/newshadows

anyways, i'll be sure to check for alternative implementations to be sure i'm doing things correctly.

as for the linear implementation - I think I managed to pack it correct. https://github.com/zz85/three.js/blob/d55bef9406a88d2195bdaec5e85e49bebc8b847a/src/renderers/WebGLShaders.js#L1961 the problem is in the reading of the depth values - it seems that the shader doing the unpacking can't access the spotlight positions? Some strange spotLightPosition not declared errors (need unprojection of the linear space using the light's camera far and near distances). Perhaps the complexity of the shaders has already built up, (since directional and spotlights are supported), so I would have to be extra careful looking inside.

[skrat]

There is some PCSS+PCF out there http://goo.gl/IT8iL but for some reason it didn't work in my Mac/ATI/Chrome combo.

[skrat]

Just checking, any planned work on this one? @alteredq ?

[alteredq]

One day, we are definitely not done yet with shadows.

[skrat]

I'm trying my hand on VSM, but I'm hitting the wall while blurring the depth map. Here's what I have so far https://gist.github.com/3952991 . To enable blurring uncomment 121,122 , I can't get EffectComposer to do the job, any help appreciated.

[zz85]

@skrat to build VSM in the ShadowMapPlugin, it might be better to handle the renderbuffers manually as it'd get rather complex to use the EffectComposer at the Renderer level, i think. I was working on it, then left it off somewhere after I encountered a couple of bugs on my side :(

[alteredq]

I'm trying my hand on VSM, but I'm hitting the wall while blurring the depth map.

Just to make sure - not really sure from a quick glance on the code - do you blur the unpacked depth? You can't do blur on RGBA encoded values.

On that note, I'm wondering if it would make sense to change shadow mapping code to use float texture instead of RGBA encoding. This would simplify things though it would be less compatible.

Similar thing for a new depth texture extension:

http://www.khronos.org/registry/webgl/extensions/WEBGL_depth_texture/

When it came out I did quick and dirty test - depth texture didn't help much with shadow map performance and it would require quite big code flow changes but it could be useful when depth is reused for multiple things (e.g. SSAO or deferred shading). Maybe once there is a more support for this in stable browsers.

Float textures though are tempting, support there is already pretty good (WebGL Stats is now down but I remember it being in high nineties of percent).

If I get to this, I would try single channel float texture. If that works, that would be cool. Full blown RGBA float would be too heavy. Last time I tried single channel regular textures, support was still sketchy but that was long ago.

[skrat]

@alteredq good point, it could be that I am, although I changed the blur shader to output just red color, then I drawn light.shadowMap on the screen and it would still be either black or white. I couldn't figure out that EffectComposer business. If you help me to get the red square from that blur shady code, then putting unpacking into blur shader is trivial.

[alteredq]

I think the problem is that you are using EffectComposer without any render pass, just trying to reuse existing render target from light.shadowMap. This confuses double buffering.

As @zz85 said, for his use case it would be probably easier just to handle render targets manually.

You could create something with EffectComposer but it would be harder to make it in an efficient way. You would need to render shadowmap into composer's target, then do blurring, then render result back into shadowmap (4 passes). Or hack somehow how double-buffering is handled and make sure you start from the right render target and end with result in the right render target.

With manual render pipeline, you would create one extra render target, have one pass doing horizontal blur reading from shadowmap writing into that target, then another vertical blur pass reading from that target and writing back into shadowmap (2 passes).

[alteredq]

So I tried using single channel float texture - it works with OpenGL but ANGLE throws tons of exceptions about incomplete framebuffer :S

All LUMINANCE, ALPHA and LUMINANCE_ALPHA formats are broken, only RGB and RGBA work.

[zz85]

@pyalot has a great article about soft shadow mapping in webgl http://codeflow.org/entries/2013/feb/15/soft-shadow-mapping/ i'm now more convinced about VSMs... would try implementing this again when i get the time if and no others attempt this.

[pyalot]

I support that idea :+1: :)

[mrdoob]

Me too! :+1:

[pyalot]

Note that I update the code for VSMs a little bit (to accomodate mobiles). I can report that everything works fine using mediump precision and with half-float textures on android.

[ghost]

Just wondering if progress has been made on VSMs.

[tranek]

@zz85 Did you ever get a chance to implement VSM?

[pyalot]

doh :)

[wlalele]

:(

[mrdoob]

Yeah... Maybe @zz85 would like to give it another go now that the WebGLRenderer has been cleaned up though 😊