High memory usage results in crashes in latest git

[VanessaE]

Hard to explain really, but for no apparent reason, the latest git has become very crashy, at least on Linux - I can go anywhere from 10-20 seconds to a minute or two. Simply walking around or flying is sufficient to trigger this bug. On the controlling terminal, all I get is:

"terminate called after throwing an instance of 'St9bad_alloc' what(): std::bad_alloc Aborted"

Link to debug.txt from the last run only (deleted before running this test):

http://digitalaudioconcepts.com/vanessa/hobbies/minetest/minetest-debug-20120618.txt

Link to strace output of this same run:

http://digitalaudioconcepts.com/vanessa/hobbies/minetest/minetest-strace-20120618.txt

(note: it's big - 32.5 MB)

[VanessaE]

On reviewing the strace log, it shows up as if there is an out-of-memory problem. According to 'top', Minetest was using 2.8GB of RAM (resident; 3.0 GB virtual). I tend to blame my 512px texture packs for some of the RAM usage, but this is more than an order of magnitude more than the pack (176 MB) and my testing world (6.8 MB) combined.

The machine in question has 4GB of RAM and 4GB of swap.

On a quick re-check, when the same thing happened again, Minetest was again using 2.8 GB, and there was 265 MB of free RAM and 3.9 GB of free swap.

Sometimes when this happens, it's more of a soft crash, in that the program stays running, but drops to a window with the menu background and a dialog reading only "Some exception:" and a "Proceed" button. If I click that, I'm taken back to the main menu. While in the main menu at this point, Minetest remains at ~2.8/3.0 GB usage. Trying to re-start the world takes me back to the "Some exception" screen.

Other times, when it crashes, it actually closes the game entirely, and the system frees the RAM that was occupied. The same is true if I just close the game after a soft crash.

On continuing to read and research, I was reminded of the memory limit in 32 bit Linux installs (of which mine is one).... which just happens to be about 3 GB. So clearly I'm hitting that upper limit. Time to upgrade to 64 bit I guess. :-/

Confirmed: it still uses over 2.9 GB but it works fine in amd64 - and in fact I seem to be getting a markedly higher frame rate than under 32 bit as well (easily 75% better).

[Uberi]

This may not be relevant, but have you tried using the video_driver = opengl option?

[VanessaE]

Wouldn't be necessary in my situation, as the game is either opengl or direct3d (or whatever it's called)....and I run Linux, so the latter isn't an option. :-)

[celeron55]

How much does Minetest consume memory after startup, with and without your texture pack and mod configuration?

[VanessaE]

Here's what I did to obtain a complete set of numbers:

Move my standard configuration folder out of Minetest's reach to force it to a completely stock configuration. Using a system-wide install of Minetest as obtained and compiled from git as usual about 21 hours ago, with no mods, no texture packs, and no changes to the default configuration.

At startup, just sitting at the main menu: 30 MB resident, 132 MB virtual After creating a new world and joining it: 144 MB resident, 833 MB virtual (noted the moment the world appeared) After standing in one place panning the camera for a few minutes to load up the map: 278 MB resident 921 MB virtual

Exit to OS....copy in my usual set of mods from the moved-out-of-the-way .minetest folder, restart the program... Still entirely on the default settings otherwise, and no special texture pack:

Startup: 30 MB resident, 132 MB virtual In world (the same one): 157 MB resident, 864 MB virtual After panning the camera for a bit: 249 MB resident, 879 MB virtual

Exit to OS... tweak minetest.conf to have the game load my 512px texture pack from its usual location... restart the program:

Startup: 31 MB resident, 132 MB virtual In-world (same one again): 3.1 GB resident, 3940 MB virtual (not sure why top used different units here) After panning: 3.1 GB resident, 3942 MB virtual.

Exit to OS....restore my standard configuration (all my usual mods, 512px texture pack, and all of my configuration preferences)... copy the test world over to it.... restart the game. Of note: my view range is set to 100 at this point, as part of my usual configuration:

Startup: 31 MB resident, 132 MB virtual In world (same one again): 2.7 GB resident, 3430 MB virtual After panning: 2.8 GB resident, 3446 MB virtual

Note that the game created a world where I'm standing on a tiny island ~6 nodes across, and surrounded mostly by water, with the only other landmasses way out near the edge of my view range.

[VanessaE]

With the latest git (fetched today) and the current version of my texture pack, and several more mods added for stress testing the game, the current git now spikes over 4GB of memory (which is all I have, so it has to dive into swap) before eventually being terminated by the Linux kernel's out-of-memory killer. The game world does not show up - it crashes while the window is still black (during what I presume is the texture-extrude process).

Links to the various files used for these tests (sizes are after extracting):

World is newly-created.

Texture pack (512px, 265 MB): https://github.com/VanessaE/512px-realistic-textures/zipball/master

(Contains a bit over 1500 individual PNG images)

Mods (everything loaded into mods/minetest, 24MB) http://digitalaudioconcepts.com/vanessa/hobbies/minetest/minetestmods-20120726.zip

(Mod ZIP file contains: add_moreores_tool growing_trees poisonivy animatedtorches homedecor pushable_block blox Jeija-minetest-mod-mesecons-04bcc45 stairsplus bobblocks junglegrass template bushes locked_sign throwing coloredwood maptools unifiedbricks cotton moreblocks unifieddyes doorsplus moreores wobblycactus drippingwater nature worldedit fireplace particles xdoors flowers pipeworks)

[Jordach]

When I am using the pack with a clean world on Windows 7 32-bit (nothing wrong) it used upto 600mb in ram, I know, even the big PC titles like Team Fortress 2 don't even break 512mb of ram.

My tests are concluded on Win 7 32Bit Home Premium.

With vanessae's full textures.

Minetest failed loading her mods + test world. The OS almost locked up, bar the task manager.

Total RAM + Swap = 1.4gb

With a minimal game mode:

Total RAM + Sawp = 1.034gb

Running Minetest default textures:

In Vanessa's mod world:

Total RAM + Swap = 700mb

In a minimal world

Total RAM + Swap = 400mb

So, yes, Minetest is leaking memory, and a large amount too.

[celeron55]

I am now guessing this is because of the per-pixel-cube-extruded wielded item meshes. Try removing all the high-resolution tool and item textures.

If it gets rid of the problem, the solution will be falling back to flat tool wield textures when the images of them are of too high resolution.

[VanessaE]

For the most part I already did that some time ago. Tools are still high-resolution since they're meant to be wielded of course, but everything meant for inventory/crafting display is 64px (even in the 512px pack). I may be imagining it, but it seems like the more images I add, even when they aren't tools or inventory/craft items, the longer the extrude process takes.

Perhaps a meet-halfway approach is best? If it's a tool, extrude it. If it's an inventory/craft item, draw it flat. For everything else, just render a cube/nodebox normally and wield the result.

[Uberi]

I don't really like that approach; sometimes we need to use craftitems as tools (matches in the fire mod, for example).

[PilzAdam]

Maybe related: #214 and #229

[sapier]

not reproducable in 0.4.8

[tenplus1]

The same bug is happening on Xanadu server regularly and we are using 16x default textures and it hardly breaks to 500mb limit for memory use out of 4gb... from 0.4.12 upwards this std:bad_alloc error always crops up for reasons unknown...

2015-04-30 12:12:35: ACTION[ServerThread]: player G1127 crafts default:stick 4 terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc

[celeron55]

@tenplus1: This issue is about client-side crashes. Yours is server-side.

Also this issue is 3 years old. Open a new one instead of commenting here.