R_Mod_CompileShadowMap doesn't calculate allocation requirements properly

[andreymal]

If r_shadow_realtime_world is 1 and the radius of the light in the attached map is greater than 336.0, DarkPlaces crashes.

It crashes randomly (sometimes during map loading, sometimes during shutdown), but as of time of writing this issue, DarkPlaces crashes during shutdown (or after r_restart) with this message:

Mem_Free: trashed tail sentinel (alloc at ../../../model_shared.c:1013, free at ../../../model_shared.c:1180)

rtltest.pk3.zip

[hemebond]

Are you sure it's the radius? I have lights with a radius of 512 and it doesn't crash. However I have another map I worked on this weekend that crashes consistently, with that same error, even with r_shadow_realtime_world 0.

I ran darkplaces-sdl debug with gdb and got this:

Mem_Alloc: pool Renderer, file ../../../gl_rmain.c:3518, size 16777216.000000 bytes (16.000000 MB)
Draw_CachePic("gfx/loading"): frame 2: loading pic
Compiling shader mode 0 permutation 5
GLSL shader glsl/combined_crc33764.glsl generic diffuse viewtint compiled (1 textures).
Compiling shader mode 0 permutation 4
GLSL shader glsl/combined_crc33764.glsl generic viewtint compiled (0 textures).
Draw_CachePic("gfx/conback"): frame 2: loading pic
Draw_CachePic("white"): frame 2: loading pic
Compiling shader mode 4 permutation 20000000
GLSL shader glsl/combined_crc33764.glsl vertexcolor alphagenvertex compiled (1 textures).
Starting render module "CL_Video"

Program received signal SIGSEGV, Segmentation fault.
__strlen_sse2 () at ../sysdeps/x86_64/multiarch/strlen-sse2.S:142
142 ../sysdeps/x86_64/multiarch/strlen-sse2.S: No such file or directory.

Not sure if it's useful or not.

[Baker7]

The trashed sentinel means memory is written outside an array. Somewhere data is written somewhere it shouldn't be.

I'm not super-familiar with this part of the code, there are arrays within arrays.

[bones-was-here]

A BT is unlikely to catch it because it's a voluntary shutdown not a crash.

[hemebond]

I sometimes get the error when loading the map. It always pops up the "engine error" dialog window when it segfaults.

I think I'm missing some source file which is why the segfault message can't find the file. If you have any idea what I need to get you a proper stacktrace let me know.

[bones-was-here]

The missing file error above is about a platform-specific implementation of the libc function strlen(), I'm not sure what would be going on there. Are you using a glibc-based distro?

Edit: DP's SIGSEGV message says "Engine Crash: Segmentation fault", so I think that's your gdb crashing there...

[hemebond]

When leaving the buggy map to load the next map, I see this error:

Halls of the Dead
BGM track 7 playing...
Connect: signon stage 1 of 4CL_SignonReply: 1
Connect: signon stage 2 of 4CL_SignonReply: 2
Mem_Alloc: pool Renderer, file ../../../gl_rmain.c:3518, size 16777216.000000 bytes (16.000000 MB)
player connected
player entered the game
Connect: signon stage 3 of 4CL_SignonReply: 3
Connect: signon stage 4 of 4CL_SignonReply: 4
Engine Error: Mem_Free: not allocated or double freed (free at ../../../model_shared.c:245)
CL_Disconnect
Sending clc_disconnect
Disconnected
SV_Shutdown
Client "player" dropped (Server shutting down)
Stopping render module "CL_Video"
Stopping render module "Models"
Engine Error: Mem_FreePool: pool already free (freepool at ../../../model_shared.c:245)
recursive shutdown

If I load a different map on start, then load the buggy map, I get this:

Maximum Speed
CDAudio: Bad track number 0.
Connect: signon stage 1 of 4CL_SignonReply: 1
Connect: signon stage 2 of 4CL_SignonReply: 2
Mem_Alloc: pool Renderer, file ../../../gl_rmain.c:3518, size 16777216.000000 bytes (16.000000 MB)
player connected
player entered the game
Connect: signon stage 3 of 4CL_SignonReply: 3
Connect: signon stage 4 of 4CL_SignonReply: 4
Engine Error: Mem_Free: not allocated or double freed (free at ../../../gl_rmain.c:2743)
CL_Disconnect
Sending clc_disconnect
Disconnected
SV_Shutdown
Client "player" dropped (Server shutting down)
Stopping render module "CL_Video"
Stopping render module "Models"
Engine Error: Mem_Free: trashed tail sentinel (alloc at ../../../model_brush.c:1704, free at ../../../model_shared.c:245)
recursive shutdown

No gdb used for either of these tests.

[bones-was-here]

Ima improve the error handling before trying to find the actual bug :P

[bones-was-here]

It should no longer crash. A workaround fixes the test map, and in case insufficient memory is allocated a warning Mod_ShadowMesh_AddMesh: insufficient memory allocated! is printed instead of crashing. Please post details if you ever see this warning, edge cases to test would be useful when developing a better implementation. Going to leave this open because there's still some work to do there (see comments in Mod_ShadowMesh_AddMesh).

[andreymal]

It doesn't crash now, and I don't see any warnings (yet)