Inverted shadows when lightning strikes at night

[NetSysFire]

It only happens at night. The lightning looks fine at day.

glxinfo:

Extended renderer info (GLX_MESA_query_renderer):
    Vendor: AMD (0x1002)
    Device: AMD Radeon RX 570 Series (POLARIS10, DRM 3.44.0, 5.16.11-arch1-2, LLVM 13.0.1) (0x67df)
    Version: 21.3.7
    Accelerated: yes
    Video memory: 8192MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2

Config (github does not like ini files, just mv it to the ini extension):

el.txt

Ping @gvissers, I hope I ping the right person since I intend to ping Grum

[pjbroad]

Is this build from latest git or another build? If one of the released builds, what's the version string?

[NetSysFire]

This is -git (last built ecff412e8b39cb0e68ded18a4cb06822751f251b). I remember this happening since a while, so it is not a regression in a recent version.

[gvissers]

Strange bug, will see if I can reproduce it, though I've never seen it before. Not related to the recent water changes, as this affects the ground. Which means, if I'm not mistaken, that this is not a shader issue, but old-style OpenGL lighting behaviour. Might be a driver bug. Only happens at night, which is when the (positional) street lights are lit, might be connected as well, during the day we only have the (directional) sun light to deal with. [EDIT: in daylight, there is no light used for the lightning, so it makes sense the shadows are not affected]

Ping @gvissers, I hope I ping the right person since I intend to ping Grum

Pong. Yeah, you've got the right person :)

[NetSysFire]

I wonder if there is an outdoor map entirely without lights where I could test that (if one exists and that would help)? Throw at me anything you want test or any other info you might need.

I am pretty sure I saw this when I first started playing EL and since I also got the water shader issues which were fixed recently (no ripples, I could see through objects and the reflection was stretched if you remember), so I wonder if my opengl version is too recent again :p

[gvissers]

You can build the client with the DEBUG option enabled, and then use Shift+Ctrl+t to simulate a lightning strike.

I've done some digging, but have not found anything so far. Things look normal over here, with shadows darker than the light from the lightning bolt. Colour values also seem to be neatly clipped, so it's not like the eye candy issue where out of bounds colour values were generated.

[NetSysFire]

https://user-images.githubusercontent.com/59517351/158068940-d6770a0c-a27e-4d02-bb27-8f450ff0ec18.mp4

Here is a video of me letting lightning strike at "night". As you can see, I also simulated a lightning strike indoors with lights -1. There was an insane amount of spam on my stdout regardless of anything I have done. el.log This is the normal client log: main_9c3f0780.log

What would be the next steps to debug this? I also tried enabling the opengl trace option but this made the client hang on login without any error. Also, interestingly enough, when I uncheck the shadow bug option in the settings, the entire ground becomes black when lightning strikes but shadows look correct otherwise.

[gvissers]

Sorry for taking so long to respond.

There was an insane amount of spam on my stdout regardless of anything I have done.

Yeah, we should remove those eye candy debug prints. They are not very useful.

I also tried enabling the opengl trace option but this made the client hang on login without any error.

Obviously, that shouldn't happen either. Can you run it through gdb and find where it hangs? Maybe open a separate issue for that.

Also, interestingly enough, when I uncheck the shadow bug option in the settings, the entire ground becomes black when lightning strikes but shadows look correct otherwise.

:raised_eyebrow: Unchecking that awfully labeled option should disable shadows. Unless I'm sorely mistaken, you should not see shadows at all. In fact, it maps one on one with the "Shadows" option in the "GFX" tab.

What would be the next steps to debug this?

Yuck, I'm getting nowhere with this. Can you try the following patch:

diff --git a/eye_candy/eye_candy.h b/eye_candy/eye_candy.h
index 334053dd..7eaf2731 100644
--- a/eye_candy/eye_candy.h
+++ b/eye_candy/eye_candy.h
@@ -205,7 +205,8 @@ namespace ec

 #ifdef DEBUG
 #ifndef EC_DEBUG
-       const int EC_DEBUG = 1;
+//     const int EC_DEBUG = 1;
+       const int EC_DEBUG = 0;
 #endif // EC_DEBUG
 #else // DEBUG
 #ifndef EC_DEBUG
diff --git a/shadows.c b/shadows.c
index 886e9648..189fce92 100644
--- a/shadows.c
+++ b/shadows.c
@@ -550,6 +550,10 @@ void setup_shadow_mapping()
                if (lightning_ambient_color[0]+0.2 > shadow_color[0]) shadow_color[0] = lightning_ambient_color[0]+0.2;
                if (lightning_ambient_color[1]+0.2 > shadow_color[1]) shadow_color[1] = lightning_ambient_color[1]+0.2;
                if (lightning_ambient_color[2]+0.2 > shadow_color[2]) shadow_color[2] = lightning_ambient_color[2]+0.2;
+printf("shadow color = %f, %f, %f, %f\n", shadow_color[0], shadow_color[1], shadow_color[2], shadow_color[3]);
+shadow_color[0] = 0.0;
+shadow_color[0] = 1.0;
+shadow_color[0] = 0.0;
        }

        glPushMatrix();

That should:

• remove the eye candy effect spam
• print the RGBA shadow colour on stdout when lightning strikes (during the night).
• after that, make the shadow color green.

If all is well, the RGB values should all be between 0 and 1, and the alpha component should be 1. I don't expect these to be out of range, but would like to rule it out as well.

[NetSysFire]

🤨 Unchecking that awfully labeled option should disable shadows. Unless I'm sorely mistaken, you should not see shadows at all. In fact, it maps one on one with the "Shadows" option in the "GFX" tab.

Here, I recorded the exact behavior. Sorry for the 8 seconds of nothing, I do not know why ffmpeg did that. I need to check its behavior at day, but it will become day soon in EL and I can test that.

https://user-images.githubusercontent.com/59517351/159212256-aac23587-d74d-4796-8185-c40ac2e0f972.mp4

el.log

And here is the output.

[gvissers]

For some reason Firefox refuses to play your videos over here (claims file is corrupt). Chrome has no issues, so I have no idea what FF's problem is.

Anyway, so the shadow bugs option does indeed disable shadows altogether, so that's good. As to why the scene turns black, I do not yet know. It's not because the light color is black, the lightning color has constant (non-zero) ambient and diffuse colors (and even when it was black, there is a non-zero ambient component, zo it should be dark but not entirely black). I am thinking the depth texture is not blended correctly with the scene somehow.

Question: does the water with the new water shader also become black when lightning strikes?

[NetSysFire]

For some reason Firefox refuses to play your videos over here (claims file is corrupt). Chrome has no issues, so I have no idea what FF's problem is.

I have the exact same problem. To make sure the uploaded mp4 works I actually manually open the link to the mp4 in mpv. Just linux problems I guess.

I am thinking the depth texture is not blended correctly with the scene somehow.

I got two more videos for you. The comparison to lightning and shadows at daylight:

https://user-images.githubusercontent.com/59517351/159261060-d8781b14-0b1c-42ba-b61a-b9b10fe573dd.mp4

So we got two problems here as far as I can see:

1. The color the lightning casts onto the ground is black at night for whatever reason.
2. The shadows caused by the lightning strike are broken, too.

Question: does the water with the new water shader also become black when lightning strikes?

Answer: It is complicated.

https://user-images.githubusercontent.com/59517351/159262481-123b3364-9ab3-4051-8326-383029190abd.mp4

As you can see, the new water shader is totally broken for me. It does not only break the water, it also breaks the sky. With the new water shader disabled, the water is not affected by those shadows and remains visible.

[gvissers]

As you can see,

Had to resort to mpv, as even Chrome would not play that file :)

the new water shader is totally broken for me.

It almost looks as if it's not using a shader at all. Anything in the log related to shaders? grep shader ~/.elc/log/main_*.log should do it.

With the new water shader disabled, the water is not affected by those shadows and remains visible.

Okay, that's what I expected. The ground tiles are rendered using classic OpenGL, whereas the water is rendered using a GLSL shader (which explicitly checks the depth buffer and blends with the light or shadow color). So somehow your drivers misinterpret how to apply the depth map. Can't say I blame them as I am getting mightily confused myself.

BTW, from the looks of the water I assume you have frame buffer support enabled. Is that correct?

[gvissers]

It almost looks as if it's not using a shader at all.

Looking at the video again, I'm pretty sure that is exactly what happens (the water height changing gives it away). Do you have the new water shader files installed? The should be a new_water.vert and new_water.frag in the shaders directory in your EL data folder.

[NetSysFire]

Anything in the log related to shaders?

The second I switch I get a spam of:

[2022-03-21 12:25:03, reflection.c:1416] Info: Using the new water shader
[2022-03-21 12:25:03, reflection.c:1040] Error: OpenGL invalid value
[2022-03-21 12:25:03, reflection.c:1136] Error: OpenGL invalid value
[2022-03-21 12:25:03, reflection.c:1023] Error: OpenGL invalid value
[2022-03-21 12:25:03, reflection.c:1028] Error: OpenGL invalid value

BTW, from the looks of the water I assume you have frame buffer support enabled. Is that correct?

Yes. At the very top of the issue there is the config file I uploaded.

Do you have the new water shader files installed?

No. Where do I get them?

[gvissers]

No. Where do I get them?

They're also included with the git sources (in the shaders directory). Just copy them over.

[NetSysFire]

That worked. But it is currently day in EL for two more hours. Is there anything I can do to change the time to night so I can test the new water shader behavior with lightning strikes at night?

So far the only difference I can see is that the reflections seem a tiny bit brighter.

[NetSysFire]

Question: does the water with the new water shader also become black when lightning strikes?

Answer: No. It appears to be illuminated.

[gvissers]

Does this make a difference?

diff --git a/shadows.c b/shadows.c
index 886e9648..87cd5c8d 100644
--- a/shadows.c
+++ b/shadows.c
@@ -585,7 +585,8 @@ void setup_shadow_mapping()
 #else
        glTexEnvi(GL_TEXTURE_ENV,GL_TEXTURE_ENV_MODE,GL_COMBINE_EXT);
        glTexEnvi(GL_TEXTURE_ENV,GL_COMBINE_RGB_EXT,GL_INTERPOLATE_EXT);
-       glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE0_RGB_EXT,GL_PREVIOUS_EXT);
+//     glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE0_RGB_EXT,GL_PREVIOUS_EXT);
+       glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE0_RGB_EXT,GL_PRIMARY_COLOR_EXT);
        glTexEnvi(GL_TEXTURE_ENV,GL_OPERAND0_RGB_EXT,GL_SRC_COLOR);
        glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE1_RGB_EXT,GL_CONSTANT_EXT);
        glTexEnvfv(GL_TEXTURE_ENV,GL_TEXTURE_ENV_COLOR,shadow_color);

[NetSysFire]

Unfortunately not.

[gvissers]

What about this? Does it

• Make everything black, or
• Show the scene without shadows (disregard the water, that is fine)
• Something else?

diff --git a/shadows.c b/shadows.c
index 886e9648..a93c6649 100644
--- a/shadows.c
+++ b/shadows.c
@@ -587,8 +587,9 @@ void setup_shadow_mapping()
        glTexEnvi(GL_TEXTURE_ENV,GL_COMBINE_RGB_EXT,GL_INTERPOLATE_EXT);
        glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE0_RGB_EXT,GL_PREVIOUS_EXT);
        glTexEnvi(GL_TEXTURE_ENV,GL_OPERAND0_RGB_EXT,GL_SRC_COLOR);
-       glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE1_RGB_EXT,GL_CONSTANT_EXT);
-       glTexEnvfv(GL_TEXTURE_ENV,GL_TEXTURE_ENV_COLOR,shadow_color);
+//     glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE1_RGB_EXT,GL_CONSTANT_EXT);
+//     glTexEnvfv(GL_TEXTURE_ENV,GL_TEXTURE_ENV_COLOR,shadow_color);
+       glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE1_RGB_EXT,GL_PREVIOUS_EXT);
        glTexEnvi(GL_TEXTURE_ENV,GL_OPERAND1_RGB_EXT,GL_SRC_COLOR);
        glTexEnvi(GL_TEXTURE_ENV,GL_SOURCE2_RGB_EXT,GL_TEXTURE);
        glTexEnvi(GL_TEXTURE_ENV,GL_OPERAND2_RGB_EXT,GL_SRC_COLOR);

Some context: If I'm not mistaken, the code is supposed to compute (in this stage) the new pixel (or light, not sure?) color as

C_p * C_d + C_s * (1 - C_d)

where C_p is the existing pixel color (without shadowing, that's SOURCE0), C_s is the shadow color (SOURCE1), and C_d (SOURCE2) is the depth map value, which I think should either 0 or 1 depending on whether the pixel is visible from the perspective of the lightning bolt. If C_d is 0, the pixel is not visible from the lightning bolt, and result becomes the shadow color; if it is 1 the light from the bolt can reach the pixel and the result becomes the existing color. I'm trying to determine if something wonky is going on with the depth map (the C_d value), or if the previous pixel color C_p is somehow not what we expect it to be (i.e. black).

Above patch replaces C_s with C_p. That way, not matter what the value of C_d, the outcome of the mixing should be C_p, i.e. you should see the normal scene without shadows (but lighted by the lightning).

[NetSysFire]

What about this?

I could not perceive any change. Sorry, I meant to post a video too but I went to bed shortly after that reply.

https://user-images.githubusercontent.com/59517351/160097503-082d3f09-4b71-4c18-87d4-3fe515968371.mp4

It is currently not nighttime in EL, but I shall test the new patch soon.

[NetSysFire]

I tested it and now everything is black, minus water (new water shader) and illuminated objects such as street lights. Logged shadow color is 0.7, 0.65, 0.75, 1.0.

By the way, I am having trouble applying your patches. The line numbers are off by at least 10 lines. I am using the latest master and both git apply or patch complain.

[gvissers]

I tested it and now everything is black, minus water (new water shader)

Ok, so it looks like the previous rendering stage is not being picked up correctly. Pfff, I don't have a solution for you right now, partly because I'm trying to figure out how this code is actually supposed to work as I go along. I will try and see if I can come up with a solution

By the way, I am having trouble applying your patches.

Probably because I revert my earlier patches from this thread, and you didn't. Best assume patches refer to vanilla master.

[NetSysFire]

Probably because I revert my earlier patches from this thread, and you didn't. Best assume patches refer to vanilla master.

But I did try a clean vanilla master, too.

I will try and see if I can come up with a solution

If you need anything else just shout at me and I will try to get you some more debugging info.

[gvissers]

Possibly related: http://www.eternal-lands.com/forum/index.php?/topic/61740-el-on-chromebook-the-night-is-black/

[NetSysFire]

Gamma does not have any effect for me either. But that might be a Xorg only thing.