Game process runs out of adressable memory

[Breeki32]

Dear @alinmear

When using mods who pile up to more than 8GB RAM allocation, the game dies after a short time unable to claim more than roughly 8GB RAM, even on hosts than have more RAM with no limitations in place sides docker or the OS.

this is that the logs look like (it dies after this shown warning, struggling to allocate more): https://imgur.com/QZBEcK5

well this kept us busy some time with out server after we recently chose to switch to AoC with our community server. the server died off reproducably and we could not find the issue, even the hoster shrugged it off, guaranteed that our machine really has 16GB ;) and we even tried on an other hoster with different infrastructure and still it died like that.

I´m not sure what it is, couldn´t find any clues but something in this container seems to have serious issues in either memory management or bitness ;) or at least you´ll have a clue.

how to reproduce? just add this modlist and start it up. give it some time, it will take rougly 20mins to die, uhh to start up. watch the RAM allocation grow to 7.63GB via htop, that is the moment it rips on most occasions.

"CONANEXILES_MODS=880454836,1159180273,2050780234,1701136207,2483184203,1629644846,2424793128,864199675,1369743238,1639336393,1938352194,1846399420,1113901982,1734383367,2447889148"

also we use 60hz

"CONANEXILESEngine/script/onlinesubsystemutils.ipnetdriver_NetServerMaxTickRate=60"

I know its many mods and less would be better, but in theory it should not die of OOM. at this moment i cannot exclude some memory leakage in the mods or issues with the constellations, but still it should at least die OOM at higher yields than half the available resources. its the only thing than ran on this server. and of course we don´t know what we´re doing with this but there is not much thing to fail with asides from having the correct mod order (which we do or at least in a relative way, but it doesn´t really matter since it dies in different orders and combinations in the same way).

we ran it on a 4 cpu 16gb ram intel xeon server with ssd with ubuntu 20.04, latest docker and compose installs. thank you so much for looking into this, like in the movies: you´re our only hope!

until then we´ll live without AoC and love your software, thanx!

edit: minor detailing, typos, address <3

[sshagent]

I'm also getting this. I've tried to setup this a few days ago, with 8 or so mods (including AoC). My server is dedicated host with 64 GB of ram and like you it just dies within the hour. Memory usage trickles up and then it just dies....never reaching any big numbers of use. I've got script monitoring memory usage every 5 minutes, and most I've seen it get to is around 15%

Whilst i wouldn't want it to use all my 64 gb of ram, if it wants to use more than 10 or so...then I'm fine with it. As with above, is there some limit amongst the stack that causes this?

Any help much appreciated.

edit oh and just to add I'm not using ipnetdriver_NetServerMaxTickRate=30 (default from the docker options)

.

[sshagent]

I've gradually been working through this, removing a couple of mods and starting it up. Each time the memory in use when the crash happens is lower. Always seems to be around the hour mark (maybe there is some kind housekeeping that triggers on memory clean up perhaps ).
Last go i removed AoC and the other AoC mod and no crashes thus far. If this works overnight, i'll try adding in all the other mods (and not AoC) and maybe that indicates a smoking gun

[Breeki32]

we tried the same but no other mod did trigger that RAM usage so reproduction was only possible with AoC.

for me it was a magic RAM threshhold which lets the game server fail. but the container keeps running, eating up and blocking a whole cpu. I`d add that any given combination runs flawlessly on a native windows system and scales there as you would expect, of course. The addon worked fine there.

[sshagent]

I guess as you've that working on a native windows server, then that proves mod load order isn't really at fault? Thanks for the info, that suggest i can re-insert all the other mods I've been removing today :)

[Breeki32]

even with AoC being the only one it died once like this :) (but not on win), I´m not sure what the big difference is

[alinmear]

Hi Folks; please keep in mind, that this container utilizes wine abstraction layer to run the exiles server binary (windows only). Exiles has some issues on "native" windows too, but we always have to keep in mind this abstraction layer can cause issues too.

My best advice would be trying to run the setup with all mods listet in a native setup with windows. As for now i think gportal has some good offerings running exiles server for low budget. If the memory problems still occur, it has something to do with the game or mods itself, otherwise there are some issues regarding wine.

[Breeki32]

I know, I use your setup for reasons still. It has almost no implications on the platform (others than this), is portable across common cloud hosters and has a 100% configurability through one single compose file. I did not even go on about the other features.

Sad though that AoC fans can´t get to enjoy this on cloud servers (or wsl setups because ease-of-use ) :( I personally prefer cloud services to closed shop hosters. This works fine most of the time, especially for games where there are such great management solutions like this GamingServer as a Configuration (GSaaC) 🥇 AoC is a great mod and this would be a great setup to run it with.

I didn´t expect you to bear support for wine, especially if its a known wine quirks, just asked for advice maybe if you knew which knobs and buttons I could press to change something - I couldn´t sadly :(

Thank you anyways and have a great day <3

[sshagent]

@Breeki32 have you tried this without the server tweak variables? Wondering perhaps if those enhancements push the wine / Conan / AoC too far?

[GrzegorzSeb]

I tryed that without env variables and still crash. For 100% server isn't taking more ram than 8Gb even when I have like 96 GB free... Nothing is limited. Maybe update docker image to new wine version?

[ddouglas87]

Maybe update docker image to new wine version?

I've tried winehq-devel (wine-7.8) and have the same issue.