search

gemrb / gemrb

GemRB is a portable open-source implementation of Bioware’s Infinity Engine.
https://gemrb.org
GNU General Public License v2.0
983 stars 184 forks source link

Memory leak somewhere #2188

Closed MarcelHB closed 1 month ago

MarcelHB commented 1 month ago

Bug description

Playing around in IWD2 AR620x (probably unrelated), I started to notice that the process had reached 14GB of allocated RAM, growing steadily. Need to investigate later.

GemRB version (check as many as you know apply)

MarcelHB commented 1 month ago

Hm, various smaller stuff on Valgrind, but not the dimension I observed on Windows earlier. Will continue keeping an eye on this. Yet we should have a look at the things below.

==7207== 48 (40 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 2,250 of 3,209
==7207==    at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==7207==    by 0x6D6F9AB: GetKitSpell(GemRB::FixedSizeString<8ul, &strncasecmp> const&, std::vector<SpellEntry*, std::allocator<SpellEntry*> >&) (CREImporter.cpp:431)
==7207==    by 0x6D6FC60: InitSpellbook() (CREImporter.cpp:451)
==7207==    by 0x6D6FD9D: GemRB::CREImporter::CREImporter() (CREImporter.cpp:457)
==7207==    by 0x6DB56C3: void std::_Construct<GemRB::CREImporter>(GemRB::CREImporter*) (stl_construct.h:119)
==7207==    by 0x6DB5428: construct<GemRB::CREImporter> (alloc_traits.h:661)
==7207==    by 0x6DB5428: std::_Sp_counted_ptr_inplace<GemRB::CREImporter, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<>(std::allocator<void>) (shared_ptr_base.h:604)
==7207==    by 0x6DB515A: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<GemRB::CREImporter, std::allocator<void>>(GemRB::CREImporter*&, std::_Sp_alloc_shared_tag<std::allocator<void> >) (shared_ptr_base.h:971)
==7207==    by 0x6DB4E8B: std::__shared_ptr<GemRB::CREImporter, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>>(std::_Sp_alloc_shared_tag<std::allocator<void> >) (shared_ptr_base.h:1712)
==7207==    by 0x6DB4C9A: std::shared_ptr<GemRB::CREImporter>::shared_ptr<std::allocator<void>>(std::_Sp_alloc_shared_tag<std::allocator<void> >) (shared_ptr.h:464)
==7207==    by 0x6DB2A2D: std::shared_ptr<std::enable_if<!std::is_array<GemRB::CREImporter>::value, GemRB::CREImporter>::type> std::make_shared<GemRB::CREImporter>() (shared_ptr.h:1010)
==7207==    by 0x6DAE272: std::shared_ptr<std::enable_if<!std::is_abstract<GemRB::CREImporter>::value, GemRB::CREImporter>::type> GemRB::MakeImporter<GemRB::CREImporter>() (Plugin.h:94)
==7207==    by 0x6DB2AFC: GemRB::ImporterPlugin<GemRB::CREImporter>::ImporterPlugin() (ImporterPlugin.h:30)
==7207==    by 0x6DB2B53: void std::_Construct<GemRB::ImporterPlugin<GemRB::CREImporter>>(GemRB::ImporterPlugin<GemRB::CREImporter>*) (stl_construct.h:119)
==7207==    by 0x6DA7515: construct<GemRB::ImporterPlugin<GemRB::CREImporter> > (alloc_traits.h:661)
==7207==    by 0x6DA7515: std::_Sp_counted_ptr_inplace<GemRB::ImporterPlugin<GemRB::CREImporter>, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<>(std::allocator<void>) (shared_ptr_base.h:604)
==7207==    by 0x6D9F95C: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<GemRB::ImporterPlugin<GemRB::CREImporter>, std::allocator<void>>(GemRB::ImporterPlugin<GemRB::CREImporter>*&, std::_Sp_alloc_shared_tag<std::allocator<void> >) (shared_ptr_base.h:971)
==7207==    by 0x6D981C9: std::__shared_ptr<GemRB::ImporterPlugin<GemRB::CREImporter>, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>>(std::_Sp_alloc_shared_tag<std::allocator<void> >) (shared_ptr_base.h:1712)
==7207==    by 0x6D945FA: std::shared_ptr<GemRB::ImporterPlugin<GemRB::CREImporter> >::shared_ptr<std::allocator<void>>(std::_Sp_alloc_shared_tag<std::allocator<void> >) (shared_ptr.h:464)
==7207==    by 0x6D8FD95: std::shared_ptr<std::enable_if<!std::is_array<GemRB::ImporterPlugin<GemRB::CREImporter> >::value, GemRB::ImporterPlugin<GemRB::CREImporter> >::type> std::make_shared<GemRB::ImporterPlugin<GemRB::CREImporter>>() (shared_ptr.h:1010)
==7207==    by 0x6D8A607: GemRB::CreatePlugin<GemRB::ImporterPlugin<GemRB::CREImporter> >::func() (plugindef.h:116)
==7207==    by 0x4D8E45B: GemRB::PluginMgr::GetPlugin(unsigned long) const (PluginMgr.cpp:38)
==7207==    by 0x6F101FF: std::shared_ptr<GemRB::ImporterPlugin<GemRB::ActorMgr> > GemRB::MakePluginHolder<GemRB::ImporterPlugin<GemRB::ActorMgr> >(unsigned long) (PluginMgr.h:137)
==7207==    by 0x6F0C817: std::shared_ptr<GemRB::ImporterPlugin<GemRB::ActorMgr> > GemRB::MakeImporterPluginHolder<GemRB::ActorMgr>(unsigned long) (PluginMgr.h:142)
==7207==    by 0x6F0922A: std::shared_ptr<GemRB::ActorMgr> GemRB::GetImporter<GemRB::ActorMgr>(unsigned long) (PluginMgr.h:147)
==7207==    by 0x6EFB684: GemRB::GAMImporter::LoadGame(GemRB::Game*, int) (GAMImporter.cpp:193)
==7207==    by 0x4CFBF05: GemRB::Interface::LoadGame(std::shared_ptr<GemRB::SaveGame>, int) (Interface.cpp:2450)
==7207==    by 0x4CF1C92: GemRB::Interface::HandleFlags() (Interface.cpp:792)
==7207==    by 0x4CF2E3C: GemRB::Interface::Main() (Interface.cpp:983)
==7207==    by 0x10EA8D: main (GemRB.cpp:67) 
==7207== 408 (208 direct, 200 indirect) bytes in 1 blocks are definitely lost in loss record 3,034 of 3,209
==7207==    at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==7207==    by 0x4CC4179: GemRB::GameScript::ReadResponse(GemRB::DataStream*) (ParseBCS.cpp:249)
==7207==    by 0x4CC3F9E: GemRB::GameScript::ReadResponseSet(GemRB::DataStream*) (ParseBCS.cpp:221)
==7207==    by 0x4CC3E6A: GemRB::GameScript::ReadResponseBlock(GemRB::DataStream*) (ParseBCS.cpp:207)
==7207==    by 0x4CB0F51: GemRB::GameScript::CacheScript(GemRB::FixedSizeString<8ul, &strncasecmp> const&, bool) (GameScript.cpp:1790)
==7207==    by 0x4CB0D8A: GemRB::GameScript::GameScript(GemRB::FixedSizeString<8ul, &strncasecmp> const&, GemRB::Scriptable*, int, bool) (GameScript.cpp:1757)
==7207==    by 0x4E3C035: GemRB::Scriptable::SetScript(GemRB::FixedSizeString<8ul, &strncasecmp> const&, int, bool) (Scriptable.cpp:154)
==7207==    by 0x9901B84: GemRB::AREImporter::GetActor(GemRB::DataStream*, std::shared_ptr<GemRB::ActorMgr>, GemRB::Map*) const (AREImporter.cpp:1191)
==7207==    by 0x99041D1: GemRB::AREImporter::GetMap(GemRB::FixedSizeString<8ul, &strncasecmp> const&, bool) (AREImporter.cpp:1562)
==7207==    by 0x4C3F47C: GemRB::Game::LoadMap(GemRB::FixedSizeString<8ul, &strncasecmp> const&, bool) (Game.cpp:851)
==7207==    by 0x4C3E70D: GemRB::Game::GetMap(GemRB::FixedSizeString<8ul, &strncasecmp> const&, bool) (Game.cpp:671)
==7207==    by 0x4BD8DC4: GemRB::GameControl::ChangeMap(GemRB::Actor const*, bool) (GameControl.cpp:2573)
==7207==    by 0x4CF1ECA: GemRB::Interface::HandleFlags() (Interface.cpp:815)
==7207==    by 0x4CF2E3C: GemRB::Interface::Main() (Interface.cpp:983)
==7207==    by 0x10EA8D: main (GemRB.cpp:67)
==7207==
==7207== 448 bytes in 2 blocks are definitely lost in loss record 3,036 of 3,209
==7207==    at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==7207==    by 0x69D1A05: GemRB::EFFImporter::GetEffectV1() (EFFImporter.cpp:83)
==7207==    by 0x69D1996: GemRB::EFFImporter::GetEffect() (EFFImporter.cpp:69)
==7207==    by 0x67B196D: GemRB::ITMImporter::GetFeature(GemRB::Item const*) (ITMImporter.cpp:361)
==7207==    by 0x67B1294: GemRB::ITMImporter::GetItem(GemRB::Item*) (ITMImporter.cpp:260)
==7207==    by 0x4C59DF8: GemRB::GameData::GetItem(GemRB::FixedSizeString<8ul, &strncasecmp> const&, bool) (GameData.cpp:178)
==7207==    by 0x4CFF8E4: GemRB::Interface::ResolveRandomItem(GemRB::CREItem*) const (Interface.cpp:3272)
==7207==    by 0x4CFF05D: GemRB::Interface::ReadItem(GemRB::DataStream*, GemRB::CREItem*) const (Interface.cpp:3120)
==7207==    by 0x4CFEF4D: GemRB::Interface::ReadItem(GemRB::DataStream*) const (Interface.cpp:3107)
==7207==    by 0x6D73C0F: GemRB::CREImporter::ReadInventory(GemRB::Actor*, unsigned long) (CREImporter.cpp:1121)
==7207==    by 0x6D723B8: GemRB::CREImporter::GetActor(unsigned char) (CREImporter.cpp:924)
==7207==    by 0x6EFCFC9: GemRB::GAMImporter::GetActor(std::shared_ptr<GemRB::ActorMgr> const&, bool) (GAMImporter.cpp:461)
==7207==    by 0x6EFB6F7: GemRB::GAMImporter::LoadGame(GemRB::Game*, int) (GAMImporter.cpp:196)
==7207==    by 0x4CFBF05: GemRB::Interface::LoadGame(std::shared_ptr<GemRB::SaveGame>, int) (Interface.cpp:2450)
==7207==    by 0x4CF1C92: GemRB::Interface::HandleFlags() (Interface.cpp:792)
==7207==    by 0x4CF2E3C: GemRB::Interface::Main() (Interface.cpp:983)
==7207==    by 0x10EA8D: main (GemRB.cpp:67) 
==7207== 1,504 (8 direct, 1,496 indirect) bytes in 1 blocks are definitely lost in loss record 3,120 of 3,209
==7207==    at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==7207==    by 0x4C2373D: std::__new_allocator<GemRB::ViewScriptingRef*>::allocate(unsigned long, void const*) (new_allocator.h:151)
==7207==    by 0x4C22658: allocate (alloc_traits.h:482)
==7207==    by 0x4C22658: std::_Vector_base<GemRB::ViewScriptingRef*, std::allocator<GemRB::ViewScriptingRef*> >::_M_allocate(unsigned long) (stl_vector.h:381)
==7207==    by 0x4C215E5: void std::vector<GemRB::ViewScriptingRef*, std::allocator<GemRB::ViewScriptingRef*> >::_M_realloc_insert<GemRB::ViewScriptingRef* const&>(__gnu_cxx::__normal_iterator<GemRB::ViewScriptingRef**, std::vector<GemRB::ViewScriptingRef*, std::allocator<GemRB::ViewScriptingRef*> > >, GemRB::ViewScriptingRef* const&) (vector.tcc:459)
==7207==    by 0x4C20511: std::vector<GemRB::ViewScriptingRef*, std::allocator<GemRB::ViewScriptingRef*> >::push_back(GemRB::ViewScriptingRef* const&) (stl_vector.h:1292)
==7207==    by 0x4C1F181: GemRB::View::AssignScriptingRef(unsigned long, GemRB::FixedSizeString<15ul, &strncasecmp> const&) (View.cpp:950)
==7207==    by 0x4BC95A8: GemRB::RegisterScriptableControl(GemRB::Control*, unsigned long, GemRB::ControlScriptingRef const*) (GUIScriptInterface.cpp:93)
==7207==    by 0x9B2487D: GemRB::CHUImporter::GetWindow(unsigned long) const (CHUImporter.cpp:479)
==7207==    by 0x4C2B3D6: GemRB::WindowManager::LoadWindow(unsigned long, GemRB::FixedSizeString<15ul, &strncasecmp> const&, GemRB::Window::WindowPosition) (WindowManager.cpp:107)
==7207==    by 0x6F7CAE9: GemRB_LoadWindow(_object*, _object*) (GUIScript.cpp:552)
==7207==    by 0x726EE67: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0)
==7207==    by 0x72157AE: _PyObject_MakeTpCall (in /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0)
==7207==    by 0x71B05ED: _PyEval_EvalFrameDefault (in /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0)
==7207==    by 0x6FB34BF: GemRB::GUIScript::RunFunction(char const*, char const*, _object*, bool) (GUIScript.cpp:13526)
==7207==    by 0x6FB3325: GemRB::GUIScript::RunFunction(char const*, char const*, std::vector<GemRB::ScriptEngine::Parameter, std::allocator<GemRB::ScriptEngine::Parameter> > const&, bool) (GUIScript.cpp:13491)
==7207==    by 0x4DCECCF: GemRB::ScriptEngine::RunFunction(char const*, char const*, bool) (ScriptEngine.cpp:55)
==7207==    by 0x4CF1E87: GemRB::Interface::HandleFlags() (Interface.cpp:810)
==7207==    by 0x4CF2E3C: GemRB::Interface::Main() (Interface.cpp:983)
==7207==    by 0x10EA8D: main (GemRB.cpp:67) 
==7207== 27,360 (592 direct, 26,768 indirect) bytes in 1 blocks are definitely lost in loss record 3,200 of 3,209
==7207==    at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==7207==    by 0x4DB14B4: GemRB::ProjectileServer::GetProjectile(unsigned long) (ProjectileServer.cpp:155)
==7207==    by 0x4DB128D: GemRB::ProjectileServer::GetProjectileByIndex(unsigned long) (ProjectileServer.cpp:131)
==7207==    by 0x4D430A3: GemRB::Item::GetProjectile(GemRB::Scriptable*, int, GemRB::Point const&, int, int) const (Item.cpp:250)
==7207==    by 0x4DFB8B8: GemRB::Actor::UseItem(unsigned int, int, GemRB::Scriptable const*, unsigned int, int) (Actor.cpp:9527)
==7207==    by 0x4DF21C4: GemRB::Actor::PerformAttack(unsigned int) (Actor.cpp:7312)
==7207==    by 0x4CA3160: GemRB::AttackCore(GemRB::Scriptable*, GemRB::Scriptable*, int) (GSUtils.cpp:1588)
==7207==    by 0x4C8E649: GemRB::GameScript::AttackReevaluate(GemRB::Scriptable*, GemRB::Action*) (Actions.cpp:5353)
==7207==    by 0x4CB2CE0: GemRB::GameScript::ExecuteAction(GemRB::Scriptable*, GemRB::Action*) (GameScript.cpp:2257)
==7207==    by 0x4E3D378: GemRB::Scriptable::ProcessActions() (Scriptable.cpp:513)
==7207==    by 0x4E3C2BA: GemRB::Scriptable::Update() (Scriptable.cpp:199)
==7207==    by 0x4D59749: GemRB::Map::UpdateScripts() (Map.cpp:771)
==7207==    by 0x4C42836: GemRB::Game::UpdateScripts() (Game.cpp:1577)
==7207==    by 0x4CF8078: GemRB::Interface::GameLoop() (Interface.cpp:1798)
==7207==    by 0x4CF2F4D: GemRB::Interface::Main() (Interface.cpp:999)
==7207==    by 0x10EA8D: main (GemRB.cpp:67)
==7207== 32,980 (592 direct, 32,388 indirect) bytes in 1 blocks are definitely lost in loss record 3,201 of 3,209
==7207==    at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==7207==    by 0x4DB12B4: GemRB::ProjectileServer::ReturnCopy(unsigned long) (ProjectileServer.cpp:136)
==7207==    by 0x4DB13A7: GemRB::ProjectileServer::GetProjectile(unsigned long) (ProjectileServer.cpp:144)
==7207==    by 0x4DB128D: GemRB::ProjectileServer::GetProjectileByIndex(unsigned long) (ProjectileServer.cpp:131)
==7207==    by 0x4D430A3: GemRB::Item::GetProjectile(GemRB::Scriptable*, int, GemRB::Point const&, int, int) const (Item.cpp:250)
==7207==    by 0x4DFB8B8: GemRB::Actor::UseItem(unsigned int, int, GemRB::Scriptable const*, unsigned int, int) (Actor.cpp:9527)
==7207==    by 0x4DF21C4: GemRB::Actor::PerformAttack(unsigned int) (Actor.cpp:7312)
==7207==    by 0x4CA3160: GemRB::AttackCore(GemRB::Scriptable*, GemRB::Scriptable*, int) (GSUtils.cpp:1588)
==7207==    by 0x4C8E649: GemRB::GameScript::AttackReevaluate(GemRB::Scriptable*, GemRB::Action*) (Actions.cpp:5353)
==7207==    by 0x4CB2CE0: GemRB::GameScript::ExecuteAction(GemRB::Scriptable*, GemRB::Action*) (GameScript.cpp:2257)
==7207==    by 0x4E3D378: GemRB::Scriptable::ProcessActions() (Scriptable.cpp:513)
==7207==    by 0x4E3C2BA: GemRB::Scriptable::Update() (Scriptable.cpp:199)
==7207==    by 0x4D59749: GemRB::Map::UpdateScripts() (Map.cpp:771)
==7207==    by 0x4C42836: GemRB::Game::UpdateScripts() (Game.cpp:1577)
==7207==    by 0x4CF8078: GemRB::Interface::GameLoop() (Interface.cpp:1798)
==7207==    by 0x4CF2F4D: GemRB::Interface::Main() (Interface.cpp:999)
==7207==    by 0x10EA8D: main (GemRB.cpp:67)
MarcelHB commented 1 month ago

Also not too SDL2 to ignore.

==8600== 16,243 bytes in 3 blocks are possibly lost in loss record 2,984 of 3,007
==8600==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==8600==    by 0x939992C: GemRB::DecodeRLEData(unsigned char const*, GemRB::Size const&, unsigned int) (RLE.h:31)
==8600==    by 0x9394466: GemRB::SDLVideoDriver::CreateSprite(GemRB::Region const&, void*, GemRB::PixelFormat const&) (SDLVideo.cpp:243)
==8600==    by 0x9970F55: GemRB::BAMImporter::GetFrameInternal(GemRB::FrameEntry const&, bool, unsigned char*) const (BAMImporter.cpp:163)
==8600==    by 0x9971C8F: GemRB::BAMImporter::GetAnimationFactory(GemRB::FixedSizeString<8ul, &strncasecmp> const&, bool) (BAMImporter.cpp:266)
==8600==    by 0x4C5ACCC: GemRB::GameData::GetFactoryResource(GemRB::FixedSizeString<8ul, &strncasecmp> const&, unsigned long, bool) (GameData.cpp:346)
==8600==    by 0x4B7C75D: std::shared_ptr<GemRB::AnimationFactory const> GemRB::GameData::GetFactoryResourceAs<GemRB::AnimationFactory const>(GemRB::FixedSizeString<8ul, &strncasecmp> const&, unsigned long, bool) (GameData.h:142)
==8600==    by 0x4DA354F: GemRB::Projectile::CreateAnimations(GemRB::FixedSizeString<8ul, &strncasecmp> const&, unsigned char) (Projectile.cpp:59)
==8600==    by 0x4DA43AB: GemRB::Projectile::Setup() (Projectile.cpp:292)
==8600==    by 0x4DA8172: GemRB::Projectile::Update() (Projectile.cpp:1237)
==8600==    by 0x4D5E736: GemRB::Map::UpdateProjectiles() (Map.cpp:1800)
==8600==    by 0x4CDA172: GemRB::GlobalTimer::Update() (GlobalTimer.cpp:184)
==8600==    by 0x4CFB55E: GemRB::Interface::GSUpdate(bool) (Interface.cpp:2330)
==8600==    by 0x4CF7FF0: GemRB::Interface::GameLoop() (Interface.cpp:1789)
==8600==    by 0x4CF2F4D: GemRB::Interface::Main() (Interface.cpp:999)
==8600==    by 0x10EA8D: main (GemRB.cpp:67)
MarcelHB commented 1 month ago

Strange. Valgrind Memcheck shows only above, Massif shows a good heap profile with nothing unusual.

Windows tools vary, MTuner looks good (except maybe a high amount of allocations and deallocations) whereas process explorers explode.