Regarding the memory optimization

[maxhenkes]

Heya,

sorry I haven't made this sooner but I was rather busy the last day. I'd like to take you up on your offer regarding the bad memory management :)

What kind of information do you need from me to be able to help me? :)

[maxhenkes]

https://github.com/BluSunrize/ImmersiveEngineering/issues/1533

[mindforger]

after the 5th edit: phewww quiet funny how much informations made it into my deep memory since the last time i dealt with this stuff, i corrected a few arguments as i had misunderstood them the last time i wrote about this :D

i was on business trip and just came back :)

-XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=8 -XX:+UseCompressedOops -XX:+AggressiveOpts -XX:+CMSScavengeBeforeRemark -Xms12G -Xmx16G -XX:NewSize=4G -d64 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSIncrementalMode -XX:GCTimeRatio=10 -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSIncrementalSafetyFactor=70 -XX:+CMSClassUnloadingEnabled -XX:+DisableExplicitGC -XX:MaxTenuringThreshold=4

-Xms12G -Xmx16G -XX:NewSize=4G adjust ms and mx according to your aviable RAM NewSize should be as large as required but to understand what reuired means, you'll have to bear to read the textwall of doom following XD

to sum it up: 1) small lags (like slightly but constantly stuttering sky movement) indicate a high load in the young generation memory | increase -XX:NewSize then a bit 2) additional quiet bigger laggs coming more frequently not permanently indicate a too small young generation space | decrease -XX:SurvivorRatio (minimum 1) 3) long lag spikes between 10ish ore more seconds indicate an short lifetime bleed over or a too small old gen space | decrease further -XX:SurvivorRatio or -XX:NewSize or increase total memory BUT also ask your members to explore unexplored world a bit slower ... this can not be countered in any way!

PS: add -XX:+PrintTenuringDistribution to get a view of how much of your young generation space is used during garbage collecting, you will get a hang of tuning with this options, i guaranty :)

now to the detailed explanation to give you an understandig of what I have found out so far ... i am not an expert i was just frustrated with all those heavy technical articles and "suck it bitch" articles which drop you a line without explanation

java splits the memory into sections to store all used objects based on their living time. from your log i saw that your tenuring space the one for ageing objects became quiet full > 70%.

by default the garbage collector on this space is not very timing friendly and costs quiet a bit time to salvage all dead objects, so i am using an alternative methode which will unfortunately be removed in future ... hopefully with a good replacement

objects of very short living nature are only living in "eden" space and are either beeing promoted or vaporized after usage (your eden space is constantly full causing unnecessary promotions to the more laggy default garbage collector of young generation)

and finally there is an old generation space for long living data, this is the most lagging garbage compiler working there as it is not able to "partially" clean out just a full garbage collect can work here (there were unfortunately no info about your old space, so i guess it remained pretty much untouched and is far big enough to not require any tuning)

NOW THE IMPORTANT STUFF

Java assigns the space for tenuring/ageing objects and eden with a likely fixed size designed for applications running with at most 2GB RAM

so whenever your server is growing in memory it bleeds over to the old gen and that is causing tick lag that becomes worse the more short living objects bleeds over

i could give you an extended overview of memory fragmentation but i will compress it to this: every object has a dedicated size, removing it from memory does not guarantee the same space is used again for a similar object, so space becomes like a sift and big objects wont fit, resulting in a full garbage collect and a heavy lag!

so i tuned it to use wayyyy more tenuring and eden space and a more efficient garbage collector with self adjusting collecting size, sorting for 4 generations of ageing before promoting to old space and doing small garbage collections more frequently

if 4 generations is not enough and you suffer very long single lags ever y 10ish seconds, you can increase the maxage by 2 and check back the game performance, do not go over age 10 and ask back if the long lag every 10ish seconds persist then

[maxhenkes]

Heya sorry for not getting back at you earlier but I was out of town with a shitty laptop and haven't had the time. Very nice and detailed read up. Some of the stuff I've been hearing about in my Uni class but we haven't gotten this indepth yet, very interesting to read! Thanks a lot!

I will be trying this out over the weekend and see if I can get the server to perform better and report back! :)

Again sorry that I can't really give this awesome post a proper reply but I'm a little short on time with class and all :)

[mindforger]

nah all gud, why did you enlist to something like IT XD i made this mistake and i tell everybody unless you REALLY want to do programming and developing do not study this, when you want to become an IT admin god, you just need to travel around visit workshops and collect certificates

get yourself a real company or at least some sort of science insitute to start with your final thesis, do not write something inside uni or something on your own unless it is pure gold and you make it to money once finished

[maxhenkes]

Eh I've been managing servers and websites for my self for years and it's the most boring thing ever :P I'm in IT to study programming and developing not to become an IT admin :P. I've been slowly getting back into java because it's the 2nd language we are required to learn here (4th semester) and now I'm slooowllly working on a few mods just because I can xD

[mindforger]

I'd need to validate all my gathered informations then I'd blog this somewhere, because github isn't the optimal place for this text XD maybe you got a competent tutor and can ask him to validate it ?

i mean validate my explanations not the stuff around minecraft and laggs XD

[maxhenkes]

I'd need to rewrite it to remove the minecraft parts and then see if I can get one of my tutors... However our Uni is mostly focused on C# and not so much on java, but I can try!

[maxhenkes]

Could you give me a hand at reading the output here? http://i.imgur.com/8WKZfdn.png

Would I be correct to assume that I need to increase the space for the youngest garbage collector? We're getting a bit of tps lag when the collector runs.

[maxhenkes]

Welp with the default values we were experiencing quite a bit of tps lag everytime the garbage collector runs, which is quite frequently. I tweaked the values a little according to your writeup, increased overall ram by 2gb and newSize by 1GB, and decreased -XX:SurvivorRatio to 4 but had even more frequent garbage collection which resulted in even more tick lag. -XX:SurvivorRatio 6 lesened it a little bit but now the server is on constant 15 tps from 20.

Not really sure where to go from here :P

[mindforger]

can't you assign more ram ?

[maxhenkes]

Isn't that just gonna end up with more and more and more ram?

I'm currently at

java -XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=6 -XX:+UseCompressedOops -XX:+AggressiveOpts -XX:+CMSScavengeBeforeRemark -Xms12G -Xmx18G -XX:NewSize=8G -d64 -XX:+UseConcMarkSweepGC -XX:+PrintTenuringDistribution -XX:+UseParNewGC -XX:+CMSIncrementalMode -XX:GCTimeRatio=10 -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSIncrementalSafetyFactor=70 -XX:+CMSClassUnloadingEnabled -XX:+DisableExplicitGC -XX:MaxTenuringThreshold=4 -Djline.terminal=jline.UnsupportedTerminal -jar forge.jar

That's 18 GB ram for a 50 mods pack with 6 people online. At one point it's gonna have to be enough :/

[mindforger]

you can leave out "-XX:-UseAdaptiveSizePolicy -XX:SurvivorRatio=6" for more manual tuning, as the adaption algorithm is a bit slow and sometimes makes more lag than it should prevent you said earlier " increased overall ram by 2gb and newSize by 1GB" this looked odd to me

you are currently at 18GB max with 8G newsize increase -XX:MaxTenuringThreshold=4 to 6 or 8 maybe it helps, but post some of your tenuring logs for me please

Desired survivor size 161054720 bytes, new threshold 6 (max 6)

• age 1: 6886728 bytes, 6886728 total
• age 2: 3054368 bytes, 9941096 total
• age 3: 298032 bytes, 10239128 total
• age 4: 47600 bytes, 10286728 total
• age 5: 4352736 bytes, 14639464 total
• age 6: 85296 bytes, 14724760 total

as an example how they look

[maxhenkes]

Look at the picture above, I posted that log :)

But I'll try what you suggested. We've gotten rid of all of the lag atm tho so it's good now.

[mindforger]

i totally missed that one, looks very very good to me, you are nowhere near the peak, this looks more like a mod issue to me now

[maxhenkes]

Pretty sure it was the garbage collection. We were at 20 tps, GC ran, tps at 10 for a few seconds, repeat. It's better now tho. I found a mod I used back on 1.7 called AI Improvements that really helped a lot with entity logic :P

[mindforger]

jesus O_o i totally forgot over the entitys in my pack, thanks a ton i'll add that to my pack :D

[maxhenkes]

Yeah it's a little odd that they don't look at you anymore but it decreased server load by at least 60% for us :P

[maxhenkes]

So yeah now we are having the same problem again, getting TPS lag ONLY when the garbage collection runs:

http://i.imgur.com/Q51N9VN.png

(picture is the debug output)

I'm gonna attempt what you posted above and see if that helps.