heavy color flickering with LightScatteringFilter

[GoogleCodeExporter]

jMonkeyEngine 3.0 stable
found 2013-12-??
solution provided

subject: unsymmetric effect in LightScatteringFilter leading to heavy color 
flickering in one orientation away from light position.

It's especially visible if you use a dark skybox (like in outer space with some 
star noise) with a sun (e.g. white sphere).

LightScatteringFilter in postQueue() does

        if (adaptative) {
            innerLightDensity = Math.max(lightDensity - Math.max(screenLightPos.x, screenLightPos.y), 0.0f);

This code has a bug because it calculates the center the wrong way. As a 
result, in one quarter of the x/y rotation of the camera (with me it is top 
right), rotating away from the light source, the scattering effect will not 
decrease in strength thus leading to strange flashing effects which happens 
mainly because the shaders try to read the framebuffer texture far out of range 
which makes the texture clamping effect visible and some heavy color flickering.

The working solution can look like this (copied from my derivation of 
LightScatteringFilter):

    @Override
    protected void postQueue( RenderQueue queue )
    {
        getClipCoordinates(lightPosition, screenLightPos, screenLightPos11, viewPort.getCamera());
        viewPort.getCamera().getViewMatrix().mult(lightPosition, viewLightPos);        
        // here we fix the bug of LightScatteringFilter class
        innerLightDensity = getLightDensity() * (1f - Math.max( Math.abs(screenLightPos11.x / screenSpread.x), Math.abs(screenLightPos11.y / screenSpread.y) ));
        display = innerLightDensity > 0.0 && viewLightPos.z < 0;
    }

    protected Vector3f getClipCoordinates(Vector3f worldPosition, Vector3f store, Vector3f store11, Camera cam)
    {
        float w = cam.getViewProjectionMatrix().multProj(worldPosition, store11);
        store11.divideLocal(w);

        store.x = ((store11.x + 1f) * (cam.getViewPortRight() - cam.getViewPortLeft()) / 2f + cam.getViewPortLeft());
        store.y = ((store11.y + 1f) * (cam.getViewPortTop() - cam.getViewPortBottom()) / 2f + cam.getViewPortBottom());
        store.z = (store11.z + 1f) / 2f;

        return store;
    }

Notice, this code is part of my own derivation of LightScatteringFilter which 
does not take an adaptative boolean into account because it cannot be switched 
to non-adaptative. But that should be easy to add.

The shaders are perfectly ok.

Also notice, my implementation of getClipCoordinates() returns an additional 
store11 vector (where 1 means each of the 2 coords is in range -1 to 1) just 
for optimization reasons since I need this coords also outside the function.

Btw. I have a question about the screen coords calculated and passed through to 
the shaders:
This is done by
    material.setVector3("LightPosition", screenLightPos);
which in my understanding is in pixel coords according to above code, thus 
ranges from 0 to pixel width or pixel height, respectively.
The vertex shader passes it on to the fragment shader by
    lightPos=m_LightPosition.xy;
and in the fragment shader it is used like this:
    vec2 texCoo=texCoord-lightPos;
and like this:
    scaledCoord=texCoo*scale+lightPos;
which confuses me because it looks as if the fragment shader expects the 
lightPos coords to be in range 0 to 1 while in the material the "LightPosition" 
is set in screen pixels.
Out of curiousity I tried to pass screenLightPos11 but the results are 
definitely wrong.
Can someone explain please what I am misunderstanding or overlooking here? Does 
the GPU before passing the coords to the shaders convert the coords into 0 to 1 
range first in behind? Or does it have anything to do with the name 
m_LightPosition in the vertex shader while usually globals are prefixed with g 
instead of m?

If you find it imporper to ask this question at this place, please let me know 
where I can post it instead.

Original issue reported on code.google.com by romankom...@gmail.com on 11 Jan 2014 at 7:15

[GoogleCodeExporter]

Sorry, by rechecking I saw there is the screenSpread variable which I added 
newly to my derivation of LightScatteringFilter. It is defined like this:

    protected Vector2f screenSpread = Vector2f.UNIT_XY;

    /**
     Defines the relative width and height of the viewport to apply the effect to.
     1.0 is default and means take the whole width or height.
     Make it less if you want just a portion of the width or height around the light position to apply the effect to.
     Or set it higher if you want to have the effect already when the light position still is outside the viewport.
    */
    public void setScreenSpread( Vector2f screenSpread ) { this.screenSpread = screenSpread; }
    public Vector2f getScreenSpread() { return screenSpread; }

It is my try to allow the effect just in more reduced or enlarged angle of the 
camera around the light source.

Original comment by romankom...@gmail.com on 11 Jan 2014 at 7:33

[GoogleCodeExporter]

Btw copied from jME's about box

Product Version: jMonkeyEngine SDK 3.0
Java: 1.7.0_11; Java HotSpot(TM) 64-Bit Server VM 23.6-b04
Runtime: Java(TM) SE Runtime Environment 1.7.0_11-b21
System: Windows 7 version 6.1 running on amd64; Cp1252; en_US (jmonkeyplatform)

Original comment by romankom...@gmail.com on 11 Jan 2014 at 7:46

[GoogleCodeExporter]

Ah yes and this declaration was still missing.

    protected Vector3f screenLightPos11 = new Vector3f();   // coord ranges: -1 -> 1, center = 0

Sorry for the mess. It's my first contribution.

Original comment by romankom...@gmail.com on 11 Jan 2014 at 7:47

[GoogleCodeExporter]

Btw. TestLightScattering from the jME3 samples shows this effect too, but it is 
almost not recognizable because the skybox is an even blue.

From the sun you have to rotate the camera slowly diagonally towards top right. 
The cloud disappearing towards the lower left corner of the viewport shows a 
bit of color flickering.

With a dark skybox with star noise on it and a sun sphere, the bug effect 
becomes extreme flashing to bright bluish and pinkish colors.

Original comment by romankom...@gmail.com on 11 Jan 2014 at 7:53