Increasing Default Memory From 796 MB To 1024 MB Or Higher

[Cernelius]

For the next release 1.9.0.1, I suggest increasing the default memory (currently set at 796 MB) to 1024 MB or possibly better 2048 MB.

The main user case for a memory higher than 796 MB I believe is playing big / xml-heavy maps with Hard AI (for example, playing TWW with Hard AI), but it is also relevant for maps having very heavy xml.

I tend to believe by now the issues of having 1024 MB as default (few people unable to start TripleA) should be inferior to the limits of having 796 MB as default.

Years ago Veqryn increased it to 1024, but then reverted this change because a few people were unable to start TripleA.

I suppose people that would be now unable to start TripleA at 1024 MB should be sufficiently few in numbers that can be feasibly handled by just teaching, in a few places like the developer forum, how to manually lower the default RAM assigned.

Namely, just like making some stickyed topics or something (plus an additional post under the release topic) in which saying:

If you are on an old 32 bit system or, anyway, have few RAM disposable, TripleA may fail to start and/or fail to join/host games online. You can solve this problem either by:

Modify the "system.ini" file located where TripleA was installed with something like: triplea.memory.useDefault=false triplea.memory.Xmx=796 triplea.memory.onlineOnly=false

or

Run TripleA by command line with a setting like: java -Xmx796m -classpath triplea.jar games.strategy.engine.framework.GameRunner triplea.memory.set=true

Opinions?

[DanVanAtta]

I'm a bit curious of what best practices really are for setting max memory in java applications. The default value IIRC is 256 which is a bit low.

Second, there was some reasoning of the crashes at 1024. I believe it was 32 bit systems that could have that problem where they had less than some amount of memory. It would be good to find out what that scenario was, it very well might not be a problem any longer with jdk8.

Third comment, 1024 is probably excessive. The items that sit in memory are:

• graphics
• game objects

1.8 used to maintain a cache of the last 10 used sound files. That could consume 80MB of memory pretty easily. 1.9 IIRC removed that cache, if not, the sound files are 1/10th the size, so we'll consume less than 10MB (but I really do think we removed that cache as it did not help anything).

Graphics are really the thing that should be driving memory usage. The image files are heavy.

The game objects are small, we can pack billions of them in a few MB. If not then we have a design problem, the solution is not to increase memory but to fix the object modeling. Said in other words, once an XML is parsed, even if it is giant with 80k, or 400k rows, the amount of MB to store the 1M~1B objects is very small. Hence it is really graphics that drives the memory requirement.

Which then gets to the last topic and real solution. If there is benefit for increasing max memory, we should see how we are handling graphics objects better, and the usage of memory overall. Chances are not bad that we are leaking something that is causing the bad performance. Said in another way, if the game were coded properly, then we should never really need more memory than the size of a map plus a couple of MB.

Sorry for long response.. hopefully it gives some useful additional context.

[simon33-2]

1GB of memory makes it harder to have multiple instances of Triple-A running simultaneously I would imagine.

[Cernelius]

As far as I noticed, no. The OS manages the memory anyway, and if you have little it will push down the RAM for each process. If you set it at 4096 it's not like you go there automatically, or even ever, but you usually stay at 1.5 GB in normal games, and go over 2.0 GB only in heavy ones, and only after you are playing for a while. As far as I know, the Xmx just set the maximum you can hit.

[DanVanAtta]

It can also just depend on the game as well / coding. For example, some event happens and the game creates a bunch of objects. Garbage collection, the process or removing dead objects is often triggered by certain percentages capacity occupied in memory. So what we might be doing is instead trading more frequent garbage collections for less infrequent ones which are larger. It is also the case that garbage collection has nonlinear performance, the GC of 750MB could be a lot fast than a GC at 1GB.

So it's not necessarily wise to just up the memory. There are some items we should research:

• what do other java projects do for this problem? Where do they set XmX?
• what is occupying memory when we do see more than 200MB+ or so usage. The map files are worst case around that size, it's curious what else we are storing in memory. For example, are we storing triplicate map images or something? It does not seem like we should ever be using much more than 500MB. (dev team can help answer this one, but would be of help to give example steps of how to get a game to consume 500MB+ of memory)
• where do we see performance benefits, and what are the exact timing differences? Performance work in programming is one area that needs to be pretty rigorous. It's easy for humans to guess where the problem is, and often they are wrong. For example, when parsing maps the slow parts of that process were not in the locations I had expected. Can we get a list of places/times where it seems like the performance is very different? The dev team can then work to measure some timings to verify we are actually seeing differences in the places/times we expect.

[simon33-2]

One thing I've noticed with playing with this parameter, reducing it actually increases the resident size albeit reducing the total size. Perhaps garbage collection can be swapped out if its high enough, or something.

[DanVanAtta]

@Cernelius , is there any evidence that the original problem limiting us to less than 1GB is no longer present, or is that something that we would have to investigate?

[Cernelius]

As I said, the problem was only that at 1024 MB default some people were not able to start TripleA or having other issues. As I said, I'm surmising that, as now, people having problems at 1024 MB should be very few and can be taught, as I described. Anyway, this is a minor issue, since anyway there is the option for freely setting the max RAM, which is the most important thing. If you believe that 796 MB default is preferable, anyway, just close this; people can set it higher if they want to; so not a big deal.

[DanVanAtta]

@RoiEXLab could you confirm if the latest installer allows users to specify the default max memory? If that is the case, we can consider updating that default value to 1024, and/or close this issue.

[RoiEXLab]

@DanVanAtta The latest changes generate a vmoptions file with 1G as default memory which is exactly what was requested.