Ran out of available textures in the pool when long range teleporting

NetSysFire commented 2 years ago

Describe the bug

When I long range teleport, there is a chance I get the aforementioned error.

Steps to reproduce

This is tricky to reproduce. It only happens sometimes and only when I long range teleport.

Go somewhere with many monsters (I presume this might be important)
Long range teleport
Witness a freeze. The minimap disappears and there is only blank, black space. The tiles also look rather dark.

 DEBUG    : Ran out of available textures in the pool.

 FUNCTION : SDL_Texture_Ptr pixel_minimap::shared_texture_pool::request_tex(size_t&)
 FILE     : src/pixel_minimap.cpp
 LINE     : 142
 VERSION  : 0.F

As you can see, this is clearly SDL/tiles only. There is no crash log generated but it segfaults and thus produces a coredump (thanks to systemd-coredump):

systemd-coredump[1359391]: [🡕] Process 1323152 (cataclysm-tiles) of user 1000 dumped core.

                                                     Module linux-vdso.so.1 with build-id 811cced58219753108395fea9fe694289604dc4e
                                                     Module libresolv.so.2 with build-id c915c72668282861a813f7ea3c0780f37b681dc0
                                                     Module libogg.so.0 with build-id ca0cae30a809f10db1f16e54a23b4e5ff7b2c54c
                                                     Module libvorbis.so.0 with build-id ad0a6c1af8f219b4e314090b6e8847223d424cbe
                                                     Module libopus.so.0 with build-id 649d3ddab8764d7caf73648004a35cb13e23a5e1
                                                     Module libFLAC.so.8 with build-id e568dc3d434723c72d6d302eb0b2559f27e91faa
                                                     Module libvorbisenc.so.2 with build-id 28ed815b8b1d322e6bf2bfdce4b0af2789b74335
                                                     Module libasyncns.so.0 with build-id 6acff139d56b5139ce5eb50a66c62b215e1d332d
                                                     Module libsndfile.so.1 with build-id 72d5531e74ed3e9afe49e95e449c40cb8984a258
                                                     Module libpulsecommon-15.0.so with build-id a2e110b7a0b3bb46e42515afd55a61d78185835b
                                                     Module libpulse.so.0 with build-id 10bd0ce0cfbb9405cee189faa1830cf34cab61a8
                                                     Module libpulse-simple.so.0 with build-id 5891a9b3e33b7dd638e4ca6839b837e42343c8e2
                                                     Module libpciaccess.so.0 with build-id 9dd24e76ebc38465541313b36446ffb4af842c12
                                                     Module libdrm_nouveau.so.2 with build-id ee16fa6a4d0bd08f5448cfc046ccb76b49aaa8a3
                                                     Module libdrm_radeon.so.1 with build-id 5d1615e0467d0a34a5a82cbd4ebc84851e116307
                                                     Module libdrm_intel.so.1 with build-id d160331e9c642c1d36669e2ffd875689f9186ea9
                                                     Module i915_dri.so with build-id b8a2b316ff6eaa74d7f082ce287b6502990d6c7f
                                                     Module libxcb-xfixes.so.0 with build-id a6b197ace5b9b59f913f5969eb419a88d1194f47
                                                     Module libxshmfence.so.1 with build-id 8876d9ccf620858795724ca24b9e567585a77cec
                                                     Module libxcb-sync.so.1 with build-id dda14591103b01b1311906053bf1ca9e82ade35c
                                                     Module libxcb-present.so.0 with build-id 68f5465258750e2397b1333b3ffc01ee33caa4e1
                                                     Module libxcb-dri3.so.0 with build-id 9407a2480e09dc5a1dd9d9a0652fa8d32b328c91
                                                     Module libexpat.so.1 with build-id 16023a53e7b5bbdff92bd0bf2eebd36f9b5b879f
                                                     Module libxcb-shm.so.0 with build-id fb797f299a446f559a95afcc168227482cc800d1
                                                     Module libxcb-dri2.so.0 with build-id 2dd6e65129a809dab828a1d26215a3f7a363fcc8
                                                     Module libX11-xcb.so.1 with build-id 0db4f94d8ae31b8dc9a83f825a9171656f1e532c
                                                     Module libxcb-glx.so.0 with build-id 8ad115382ad874165523dc5b4488c99d73bbc3c4
                                                     Module libdrm.so.2 with build-id 457a1a1bd25f1d5541a10d3f16c5dc1440d77c7d
                                                     Module libglapi.so.0 with build-id 9331aed5060062bcfafd463d36506312fab59549
                                                     Module libGLX_mesa.so.0 with build-id 76cdb170e96b9ad2d09c67051436088c007f4b7b
                                                     Module libGLX.so.0 with build-id 2a08836c6e6126ce9ff4496b6aacaf29ae9b4e7b
                                                     Module libGLdispatch.so.0 with build-id 501765b3a78d668860fa54229b18107473aeda4c
                                                     Module libGL.so.1 with build-id 912ac4f37a9fa2d5abcf7a9088c9983cfe46f12a
                                                     Module libXxf86vm.so.1 with build-id 01e8243d0c6c971fa9e743e6e53d606b1cb76f6e
                                                     Module libXfixes.so.3 with build-id 0a05c7e8714522bfbdd7c0027c3e2a94965664b0
                                                     Module libXrender.so.1 with build-id 97e0b9ab6ba96ebc86527cc2b3c3078aad8616b3
                                                     Module libXcursor.so.1 with build-id eb6315bcd7526aaf51b0a1307efdc587cea0a152
                                                     Module libXext.so.6 with build-id d70f24beb4fad748d6becffdcc13e51be0a2ebfa
                                                     Module libXdmcp.so.6 with build-id 8ca0792d23c8b8b4c0864297512349292bea5955
                                                     Module libXau.so.6 with build-id 1c67764663e07bec24d8951e5fd93f4d165979ff
                                                     Module libxcb.so.1 with build-id 0d1ef11740a5daad2ee331e812a51aa6574af222
                                                     Module libX11.so.6 with build-id 5ba5798d193c0065014b8c6252a0678671c8d478
                                                     Module libgpg-error.so.0 with build-id 3801187d42c4955bd93c605451430cdf1b164e63
                                                     Module libgcrypt.so.20 with build-id db45f5d5e0f7af1e77324fea1885f974619ad268
                                                     Module libcap.so.2 with build-id f56c6cd6ad4d35053340d7ff2f8f954498796739
                                                     Module liblz4.so.1 with build-id e63600ab23b2f6997f42fac2fa56e1f02ce159a1
                                                     Module librt.so.1 with build-id 75484da2d6f1515189eefa076e0a40328834cd16
                                                     Module libsystemd.so.0 with build-id 5e0c342d74b444cce6793925b41760cf298c417c
                                                     Module libdbus-1.so.3 with build-id 74f2ab9c60512f3a93c932c3f627564d42e0b11e
                                                     Module libpcre.so.1 with build-id 845483dd0acba86de9f0313102bebbaf3ce52767
                                                     Module libbrotlicommon.so.1 with build-id a4ba3f4b4571c8272343b621da812a6e24a202a7
                                                     Module libglib-2.0.so.0 with build-id 8214b56e554b7cbead4034b6ba32c90bc8c674da
                                                     Module libgraphite2.so.3 with build-id 47761dc11e553f519cde97ed9ee985be12ccdae2
                                                     Module liblzma.so.5 with build-id 8b615460aa230708c5183f16bede67aa0437d95e
                                                     Module libzstd.so.1 with build-id 6214183688018e3fba6d66db10c2c8d0b45e0578
                                                     Module libbrotlidec.so.1 with build-id 45defc036e918e0140a72f1fbce6e7692d38241d
                                                     Module libharfbuzz.so.0 with build-id 1d9dee49dd99162c3f83eb3259c3c88c6b4cd7fe
                                                     Module libbz2.so.1.0 with build-id 919597c477c9b2cb9cdbb7745ed6494ac0e6da60
                                                     Module libwebp.so.7 with build-id 87e81472aa7ff3ed1061cfc13d6960dbe00c35ab
                                                     Module libtiff.so.5 with build-id e8d89a0fb0847641ddb75c5f03fa89d68155a75d
                                                     Module libjpeg.so.8 with build-id 879ca18a2f58c25d77ba881db3a44a5676ef6c7d
                                                     Module libpng16.so.16 with build-id 2dc0bce07f199bf983c07a05fb95a6f4af83a9b3
                                                     Module libfreetype.so.6 with build-id 26c5f833068ff72660d1975cbc2074c3eb47fad8
                                                     Module ld-linux-x86-64.so.2 with build-id 040cc3dd10461562f177df39e3be2f3704258c3c
                                                     Module libdl.so.2 with build-id 5abc547e7b0949f89f3c0e21ab0c8331a7440a8a
                                                     Module libc.so.6 with build-id 4b406737057708c0e4c642345a703c47a61c73dc
                                                     Module libgcc_s.so.1 with build-id 7f8508bb914546ada778809b64b99d234337d835
                                                     Module libm.so.6 with build-id 2b8fd1f869ecab4e0b55e92f2f151897f6818acf
                                                     Module libstdc++.so.6 with build-id 9b5eeeb149bf3c4efe787fb398b44f00507aec87
                                                     Module libz.so.1 with build-id 0c1459c56513efd5d53eb3868290e9afee6a6a26
                                                     Module libSDL2_image-2.0.so.0 with build-id 7e23cb02fa57d26c7c499f0291ae557de1c910e5
                                                     Module libSDL2_ttf-2.0.so.0 with build-id 3b4c2074b5dd4dd4c6e3c54df7e589dafcce58c1
                                                     Module libpthread.so.0 with build-id 07c8f95b4f3251d08550217ad8a1f31066229996
                                                     Module libSDL2-2.0.so.0 with build-id 063170b52bf2b65c0ccabb9a1435deb3f42735fa
                                                     Module libSDL2_mixer-2.0.so.0 with build-id d60b9c328f1d972922bd1c2c70c8280cc9e53dc1
                                                     Module cataclysm-tiles with build-id 3af3ccb58c6e0f9cac0f382485ace298ff556194
                                                     Stack trace of thread 3:
                                                     #0  0x00007f25728ffd22 raise (libc.so.6 + 0x3cd22)
                                                     #1  0x00007f25728e9862 abort (libc.so.6 + 0x26862)
                                                     #2  0x000055b7d3cdf586 _ZSt20__replacement_assertPKciS0_S0_ (cataclysm-tiles + 0x71d586)
                                                     #3  0x000055b7d3fd7765 _ZNSt8__detail9_ExecutorIPKcSaINSt7__cxx119sub_matchIS2_EEENS3_12regex_traitsIcEELb1EE6_M_dfsENS9_11_Match_modeEl (cataclysm-tiles + 0xa15765)
                                                     #4  0x000055b7d3fd7a1e _ZNSt8__detail9_ExecutorIPKcSaINSt7__cxx119sub_matchIS2_EEENS3_12regex_traitsIcEELb1EE20_M_search_from_firstEv (cataclysm-tiles + 0xa15a1e)
                                                     #5  0x000055b7d3fd82f3 _ZNSt8__detail17__regex_algo_implIPKcSaINSt7__cxx119sub_matchIS2_EEEcNS3_12regex_traitsIcEELNS_20_RegexExecutorPolicyE0ELb0EEEbT_SA_RNS3_13match_resultsISA_T0_EERKNS3_11basic_regexIT1_T2_EENSt15regex_constants15match_flag_typeE (cataclysm-tiles + 0xa162f3)
                                                     #6  0x000055b7d3fcf8c1 _Z21debug_write_backtraceRSo (cataclysm-tiles + 0xa0d8c1)
                                                     #7  0x000055b7d3fad170 n/a (cataclysm-tiles + 0x9eb170)
                                                     #8  0x000055b7d3fad3f9 n/a (cataclysm-tiles + 0x9eb3f9)
                                                     #9  0x00007f25728ffda0 __restore_rt (libc.so.6 + 0x3cda0)
                                                     #10 0x00007f25728ffd22 raise (libc.so.6 + 0x3cd22)
                                                     #11 0x00007f25728e9862 abort (libc.so.6 + 0x26862)
                                                     #12 0x000055b7d3cdf586 _ZSt20__replacement_assertPKciS0_S0_ (cataclysm-tiles + 0x71d586)
                                                     #13 0x000055b7d3fd7765 _ZNSt8__detail9_ExecutorIPKcSaINSt7__cxx119sub_matchIS2_EEENS3_12regex_traitsIcEELb1EE6_M_dfsENS9_11_Match_modeEl (cataclysm-tiles + 0xa15765)
                                                     #14 0x000055b7d3fd7a1e _ZNSt8__detail9_ExecutorIPKcSaINSt7__cxx119sub_matchIS2_EEENS3_12regex_traitsIcEELb1EE20_M_search_from_firstEv (cataclysm-tiles + 0xa15a1e)
                                                     #15 0x000055b7d3fd82f3 _ZNSt8__detail17__regex_algo_implIPKcSaINSt7__cxx119sub_matchIS2_EEEcNS3_12regex_traitsIcEELNS_20_RegexExecutorPolicyE0ELb0EEEbT_SA_RNS3_13match_resultsISA_T0_EERKNS3_11basic_regexIT1_T2_EENSt15regex_constants15match_flag_typeE (cataclysm-tiles + 0xa162f3)
                                                     #16 0x000055b7d3fcf8c1 _Z21debug_write_backtraceRSo (cataclysm-tiles + 0xa0d8c1)
                                                     #17 0x000055b7d3fad170 n/a (cataclysm-tiles + 0x9eb170)
                                                     #18 0x000055b7d3fad3f9 n/a (cataclysm-tiles + 0x9eb3f9)
                                                     #19 0x00007f25728ffda0 __restore_rt (libc.so.6 + 0x3cda0)
                                                     #20 0x000055b7d49d69d5 _ZN10ui_adaptor10invalidateERK9rectangleI5pointLi0EEb (cataclysm-tiles + 0x14149d5)
                                                     #21 0x000055b7d49d6c74 _ZN10ui_adaptorD1Ev (cataclysm-tiles + 0x1414c74)
                                                     #22 0x000055b7d3d8208a _ZNSt10unique_ptrI10ui_adaptorSt14default_deleteIS0_EED1Ev (cataclysm-tiles + 0x7c008a)
                                                     #23 0x000055b7d43afc5b _ZN9live_viewD1Ev (cataclysm-tiles + 0xdedc5b)
                                                     #24 0x000055b7d413f1d4 _ZNSt10unique_ptrI9live_viewSt14default_deleteIS0_EED1Ev (cataclysm-tiles + 0xb7d1d4)
                                                     #25 0x000055b7d40fe37f _ZN4gameD1Ev (cataclysm-tiles + 0xb3c37f)
                                                     #26 0x000055b7d41435cb _ZNSt10unique_ptrI4gameSt14default_deleteIS0_EED2Ev (cataclysm-tiles + 0xb815cb)
                                                     #27 0x00007f25729024a7 __run_exit_handlers (libc.so.6 + 0x3f4a7)
                                                     #28 0x00007f257290264e exit (libc.so.6 + 0x3f64e)
                                                     #29 0x000055b7d49204d2 n/a (cataclysm-tiles + 0x135e4d2)
                                                     #30 0x000055b7d49206ab _ZN13input_manager15get_input_eventE13keyboard_mode (cataclysm-tiles + 0x135e6ab)
                                                     #31 0x000055b7d41f596a _ZN13input_context12handle_inputB5cxx11Ei (cataclysm-tiles + 0xc3396a)
                                                     #32 0x000055b7d40f97c8 _ZN4game16handle_mouseviewER13input_contextRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (cataclysm-tiles + 0xb377c8)
                                                     #33 0x000055b7d41831fb _ZN4game16get_player_inputERNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (cataclysm-tiles + 0xbc11fb)
                                                     #34 0x000055b7d41906b8 _ZN4game13handle_actionEv (cataclysm-tiles + 0xbce6b8)
                                                     #35 0x000055b7d402cf99 _Z7do_turnv (cataclysm-tiles + 0xa6af99)
                                                     #36 0x000055b7d3bbb3d2 main (cataclysm-tiles + 0x5f93d2)
                                                     #37 0x00007f25728eab25 __libc_start_main (libc.so.6 + 0x27b25)
                                                     #38 0x000055b7d3cd874e _start (cataclysm-tiles + 0x71674e)
                                                     ELF object binary architecture: AMD x86-64

Expected behavior

It does not freeze. I remember it happened way in the past, too so I highly doubt it is a recent regression.

Screenshots

No response

Versions and configuration

OS: Linux
- OS Version: LSB Version: 1.4; Distributor ID: Arch; Description: Arch Linux; Release: rolling; Codename: n/a;
Game Version: 0.F [64-bit]
Graphics Version: Tiles
Game Language: System language []
Mods loaded: [ Dark Days Ahead [dda], Disable NPC Needs [no_npc_food], No Fungal Growth [no_fungal_growth], Bionic Professions [package_bionic_professions] ]

Built from a1f0736ce23627911f124d50967f7fe79cf40eae

Additional context

No response

hexagonrecursion commented 2 years ago

Hello. I tried to reproduce this bug but could not. Could you please upload a save file where this issue can be reproduced as well as your config folder?

OS: Linux
- OS Version: LSB Version: :core-4.1-amd64:core-4.1-noarch; Distributor ID: Fedora; Description: Fedora release 34 (Thirty Four); Release: 34; Codename: ThirtyFour;
Game Version: 4d22129 [64-bit]
Graphics Version: Tiles
Game Language: English [en]
Mods loaded: [ Dark Days Ahead [dda], Disable NPC Needs [no_npc_food] ]

mkrutov commented 2 years ago

Reproduced it.

Wasn't able to reproduce it with default population, but after cranking population to 10x and long range teleporting to nearest town two times, reproduced it.

NetSysFire commented 2 years ago

Hello. I tried to reproduce this bug but could not. Could you please upload a save file where this issue can be reproduced as well as your config folder?

Unfortunately even with save scumming I can not reproduce this reliably. But by creatng a world with a heightened population density it becomes way easier to reproduce. Thanks for the tip!

ZhilkinSerg commented 2 years ago

Should we just increase pool_size for pixel_minimap::shared_texture_pool?

NetSysFire commented 2 years ago

It works if you just manually walk up to stuff, the sudden change when you long range teleport might upset it, but I am neither a C++ nor SDL expert so I have no idea.

ZhilkinSerg commented 2 years ago

Could be long teleport not unloading submaps (and thus not releasing pixel minmap textures).

esotericist commented 10 months ago

based on discord discussions, this is still present in recent experimentals.

Procyonae commented 8 months ago

Can confirm this does happen (and on default settings) it's just rare

RenechCDDA commented 5 months ago

I tried to replicate by repeatedly teleporting a few OMTs away (one fast scroll) and noticed that resting memory usage went from ~1100MB (exodii base, lot of sprites!) to ~960 after the first teleport (open field) and then rapidly climbed as the teleports continued, despite repeatedly landing in open fields.

After about 18 teleports I was sitting at 1500MB resting usage, in an open field! With another two dozen or so I was sitting at 1900MB.

Trying to parse the profiled memory now.

RenechCDDA commented 5 months ago

I have a snapshot I took shortly after the 18 teleport mark (around 20 teleports in my estimation).

(Note: All numbers below are from VS2019's reporting of allocations in map::loadn)

The big stand-out memory usage to me is this: submap::ensure_nonuniform 0x1f452a088c0 maptile_soa 18,576 cataclysm-tiles.exe

There's 7,708 instances of submap::ensure_nonuniform, each of them at 18,576 bytes for a total memory usage of ~143MB.

There's also 26,870 instances of cata::colony<item,std::allocator<item>,unsigned short>::group::group, with >99% of them being 3,602-byte allocations for a total memory usage of ~96.7MB of memory usage.

The call tree reports that jmapgen_sealed_item::apply was sucking down around 113MB of memory. That seems very strange to me. All of that should have already been handled, done, and the memory released.

RenechCDDA commented 5 months ago

Ahaha, okay, I got it. The problem is you're landing on a monster with no available spaces. Therefore game::place_player fails, returning your actual current position. game::place_player_overmap doesn't account for this, assumptions are invalidated, bad things happen.

That's also why increasing spawn rate affects it - you're more likely to land on a monster.

Here, put this in JSON. Debug spawn a desolate barn special. Teleport to it with the debug menu. 100% reproduction.

  {
    "method": "json",
    "om_terrain": "desolatebarn",
    "type": "mapgen",
    "weight": 1000000,
    "object": {
      "predecessor_mapgen": "forest",
      "rows": [
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "    TTT                 ",
        "    TTT                 ",
        "    TTT                 ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        ",
        "                        "
      ],
      "palettes": [ "desolatebarn_palette" ],
      "flags": [ "ERASE_ALL_BEFORE_PLACING_TERRAIN" ],
      "monster": { "T": { "monster": "mon_exodii_turret" } }
    }
  }

RenechCDDA commented 5 months ago

This branch stops the crashing and places the player back on the OMT they came from, but it offsets their position due to how place_player_overmap works. This could, for example, put them into solid rock or other nasty stuff.

https://github.com/CleverRaven/Cataclysm-DDA/compare/master...RenechCDDA:Cataclysm-DDA:fixteleport

So not a complete fix. Would appreciate if someone could pull it over the finish line.

Procyonae commented 5 months ago

While whoever does that (I might take a look myself) is at it favouring tiles you can stand on would be nice, it's pretty annoying when you try to tp onto a roof and just fall multiple times

Brambor commented 1 month ago

I would prefer if the teleport failed with message "Cannot teleport there, there is X in the way". The player could decide to teleport to the neighbouring tile.

Or we could do some map reveal and the player would teleport to the tile specified.

Edit: that is what is being done in the linked PR #72887

CleverRaven / Cataclysm-DDA