search

DarkPlacesEngine / DarkPlaces

The DarkPlaces Quake engine, created by LadyHavoc. Official Git repository (replaces SVN).
https://icculus.org/twilight/darkplaces/
GNU General Public License v2.0
290 stars 43 forks source link

New Immortal mod crashes to desktop with out of memory error #189

Open hemebond opened 4 months ago

hemebond commented 4 months ago

The Immortal Lock mod crashes to desktop when you enter the first level (after start). I'm using the latest master.

Apparently even Ironwail crashes due to memory usage. vkQuake is recommended by the author.

53 memory pools, totalling 2220852746 bytes (2117.970MB)
total allocated size: 2223056920 bytes (2120.072MB)
Memory pool 0x5629dc1e1d60 has sprung a leak totalling 503018517 bytes (479.716MB)!  Listing contents...
 503015997 bytes allocated at ../../../fs.c:3499
      1024 bytes allocated at ../../../image.c:1890
       736 bytes allocated at ../../../cmd.c:1565
       736 bytes allocated at ../../../cmd.c:1565
        24 bytes allocated at ../../../cmd.c:1602
Engine Aborted: Mem_Alloc: out of memory (alloc of size 18446744073474398208.000000 (17592186044191.740MB) at ../../../model_brush.c:2996)
DarkPlaces-Quake Linux, buildstring: v20140513-1451-gd7763f6f Jul 10 2024 release
Larger snippet of start and end of error dump ``` SpawnServer: immortal Mem_Alloc: pool Temporary Memory, file ../../../fs.c:3499, size 503015997.000000 bytes (479.713437 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2270, size 23350656.000000 bytes (22.268921 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2333, size 30157968.000000 bytes (28.760880 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:3219, size 29415888.000000 bytes (28.053177 MB) Mem_Alloc: pool Temporary Memory, file ../../../fs.c:3499, size 99031611.000000 bytes (94.443904 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2127, size 99031602.000000 bytes (94.443895 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:3235, size 29415000.000000 bytes (28.052330 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2371, size 33070884.000000 bytes (31.538853 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2572, size 167235744.000000 bytes (159.488434 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2573, size 111490496.000000 bytes (106.325623 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_shared.c:976, size 546508824.000000 bytes (521.191429 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2187, size 33661981.000000 bytes (32.102567 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2989, size 91291424.000000 bytes (87.062286 MB) Mem_Alloc: pool maps/immortal.bsp, file ../../../model_brush.c:2996, size 18446744073474398208.000000 bytes (17592186044191.740234 MB) memory pool list: size name 950841k ( 950900k actual) maps/immortal.bsp (+973660871 byte change) 91291424 bytes allocated at ../../../model_brush.c:2989 ... 2 bytes allocated at ../../../cvar.c:670 17 bytes allocated at ../../../cvar.c:669 491230k ( 491235k actual) Temporary Memory (+503018517 byte change) TEMP 503015997 bytes allocated at ../../../fs.c:3499 1024 bytes allocated at ../../../image.c:1890 736 bytes allocated at ../../../cmd.c:1565 736 bytes allocated at ../../../cmd.c:1565 24 bytes allocated at ../../../cmd.c:1602 53 memory pools, totalling 2220852746 bytes (2117.970MB) total allocated size: 2223056920 bytes (2120.072MB) Memory pool 0x5629dc1e1d60 has sprung a leak totalling 503018517 bytes (479.716MB)! Listing contents... 503015997 bytes allocated at ../../../fs.c:3499 1024 bytes allocated at ../../../image.c:1890 736 bytes allocated at ../../../cmd.c:1565 736 bytes allocated at ../../../cmd.c:1565 24 bytes allocated at ../../../cmd.c:1602 memory pool list: size name 950841k ( 950900k actual) maps/immortal.bsp (+0 byte change) 4625k ( 4626k actual) q3shaders (+0 byte change) 1335k ( 1338k actual) MESH_UI (+0 byte change) 0k ( 1k actual) MESH_SCENE (+0 byte change) 11k ( 12k actual) progs/beam.mdl (+0 byte change) 98k ( 104k actual) progs/metal.mdl (+0 byte change) 15554k ( 15557k actual) maps/spinny.bsp (+0 byte change) 399k ( 400k actual) progs/dog.mdl (+0 byte change) 406k ( 407k actual) progs/enforcer.mdl (+0 byte change) 79k ( 83k actual) progs/backpack.mdl (+0 byte change) 8k ( 9k actual) progs/spike.mdl (+0 byte change) 19k ( 21k actual) progs/grenade.mdl (+0 byte change) 13k ( 14k actual) progs/missile.mdl (+0 byte change) 25k ( 26k actual) progs/bolt3.mdl (+0 byte change) 8k ( 9k actual) progs/bolt2.mdl (+0 byte change) 8k ( 9k actual) progs/bolt.mdl (+0 byte change) 13k ( 14k actual) progs/plasma.mdl (+0 byte change) 59k ( 60k actual) progs/c_lance.mdl (+0 byte change) 37k ( 38k actual) progs/v_light.mdl (+0 byte change) 29k ( 30k actual) progs/v_rock2.mdl (+0 byte change) 14k ( 16k actual) progs/v_rock.mdl (+0 byte change) 43k ( 44k actual) progs/v_nail2.mdl (+0 byte change) 45k ( 46k actual) progs/v_nail.mdl (+0 byte change) 33k ( 34k actual) progs/v_shot2.mdl (+0 byte change) 27k ( 28k actual) progs/v_shot.mdl (+0 byte change) 54k ( 55k actual) progs/c_ham.mdl (+0 byte change) 40k ( 41k actual) progs/v_axe.mdl (+0 byte change) 20k ( 22k actual) progs/gib.mdl (+0 byte change) 9k ( 10k actual) progs/h_player.mdl (+0 byte change) 158k ( 165k actual) progs/s_vp_pll.spr (+0 byte change) 505k ( 507k actual) progs/player.mdl (+0 byte change) 5449k ( 5667k actual) Server Progs (+0 byte change) 91999k ( 92024k actual) maps/start.bsp (+0 byte change) 0k ( 1k actual) FONT (+0 byte change) 220517k ( 220535k actual) texture management (+0 byte change) 3813k ( 3814k actual) client (long term memory) (+0 byte change) 7836k ( 7838k actual) client (per-level memory) (+0 byte change) 19671k ( 19689k actual) sound (+0 byte change) 0k ( 1k actual) MESH_UI (+0 byte change) 0k ( 1k actual) MESH_SCENE (+0 byte change) 293k ( 293k actual) FONTS (+0 byte change) 348786k ( 349643k actual) Renderer (+0 byte change) 1433k ( 1433k actual) server (+0 byte change) 71k ( 72k actual) collision cache (+0 byte change) 356k ( 357k actual) modelinfo (+0 byte change) 418k ( 418k actual) network connections (+0 byte change) 0k ( 1k actual) crypto (+0 byte change) 203k ( 212k actual) file management (+0 byte change) 2k ( 5k actual) commands (+0 byte change) 19k ( 41k actual) commands (+0 byte change) 28k ( 46k actual) Command buffer (+0 byte change) 2188k ( 3029k actual) Zone (+0 byte change) 491230k ( 491235k actual) Temporary Memory (+0 byte change) TEMP 53 memory pools, totalling 2220852746 bytes (2117.970MB) total allocated size: 2223056920 bytes (2120.072MB) Memory pool 0x5629dc1e1d60 has sprung a leak totalling 503018517 bytes (479.716MB)! Listing contents... 503015997 bytes allocated at ../../../fs.c:3499 1024 bytes allocated at ../../../image.c:1890 736 bytes allocated at ../../../cmd.c:1565 736 bytes allocated at ../../../cmd.c:1565 24 bytes allocated at ../../../cmd.c:1602 Engine Aborted: Mem_Alloc: out of memory (alloc of size 18446744073474398208.000000 (17592186044191.740MB) at ../../../model_brush.c:2996) DarkPlaces-Quake Linux, buildstring: v20140513-1451-gd7763f6f Jul 10 2024 release Client "player" dropped (Server aborted!) ```

There is also a very strange physics bug in the start.bsp elevator where the player will bounce and and lose a lot of life by the time they get to the bottom.

bones-was-here commented 4 months ago

The most obvious issue is signed integer overflow, easily fixed with a (size_t) cast, at which point the real problem appears: DP's PVS (at least for Q1BSP) uses several orders of magnitude more memory than vkquake's.

Adding a debug print to vkquake's Mod_DecompressVis() reveals it allocates 102348 bytes for numleafs: 818760.
DP's Mod_Q1BSP_LoadLeafs() allocates 51732336555 bytes (~49GB) for num_leafs: 818760.
DP's Mod_Q1BSP_LoadLeafs() has an allocation multiplier that other engines don't seem to need, introduced in f2fbe3c626ee6b68c329879b69701e434f368a96.
43c93c6edf2dc67b6315ebddc7a71c7f8c0597d8 may also be relevant.
@ladyhavoc could you shed some light please on why this is used (and how it can be avoided)?

hemebond commented 2 months ago

Also affects the newly released Meat Jam

49 memory pools, totalling 3055927274 bytes (2914.359MB)
total allocated size: 3058726990 bytes (2917.029MB)
Memory pool 0x55bdcdcbbd30 has sprung a leak totalling 480125809 bytes (457.884MB)!  Listing contents...
 480123289 bytes allocated at ../../../fs.c:3514
      1024 bytes allocated at ../../../image.c:1890
       736 bytes allocated at ../../../cmd.c:1463
       736 bytes allocated at ../../../cmd.c:1463
        24 bytes allocated at ../../../cmd.c:1500
Engine Aborted: Mem_Alloc: out of memory (alloc of size 18446744073525950464.000000 (17592186044240.904MB) at ../../../model_brush.c:3011)
DarkPlaces-Quake Linux v20140513-1537-g2f25af65~ Aug 11 2024 release GCC
ladyhavoc commented 2 months ago

Decompressing PVS data was done to simplify support for q3bsp which doesn't use compressed pvs, and to make it faster to check pvs in several places per frame (e.g. for dynamic lights), that said in any case that uses the majority of the data it is probably faster to decompress on access because of memory bandwidth being lower that way. I'm curious if these maps are legitimately using an absolutely gigantic number of leafs or if they're just sitting on a lot of blank space in the PVS data. Reusing the same PVS data for multiple leafs when loading q1bsp/q2bsp leafs would be an option even if it's uncompressed, but I had not encountered maps like this when I made that decision.