ShrimpCryptid / Secret-Hitler-Online

An online version of Secret Hitler, a social deduction board game.
Other
82 stars 17 forks source link

Memory Leak #61

Closed ShrimpCryptid closed 1 year ago

ShrimpCryptid commented 1 year ago

The SHO server has a set amount of RAM available to it. However, the amount of RAM used by the program seems to slowly creep upwards over the lifetime of the server, until it eventually hits the upper limit and the hosting service terminates it.

The cause of the memory leak should be profiled (dump Java memory log, etc.) and fixed. Another solution would be restarting the server on a set interval.

ShrimpCryptid commented 1 year ago

Update: I've added some crash logging that will hopefully activate the next time the server runs out of memory! Hopefully, I can get some useful information out of the heap dump.

ShrimpCryptid commented 1 year ago

Thought: SHO does a LOT of logging. Is it possible that the log/terminal session never clears old output, and that's what's taking up system memory?

Test:

ShrimpCryptid commented 1 year ago

Okay, looks like the program never did do a heap dump even when it hit the memory limit. I've recorded the rate of memory usage (it seems to increase by roughly 23 MB/day) and I'll see if the rate decreases once I remove most of the logging. There are a few other exceptions getting thrown due to the Jetty API updating that I also need to address.

ShrimpCryptid commented 1 year ago

Okay, so post-fix it looks like the rate of memory increase has dropped to 3 MB/day. I need to look into Java logging libraries but for now logging does seem to be the main culprit.