search

MrTJP / ProjectRed

Redstone Engineering
MIT License
475 stars 184 forks source link

Memory Leak still in Project Red as of v4.5.5.56 #638

Closed Vaygrim closed 7 years ago

Vaygrim commented 9 years ago

Captured Data: https://ace5852.com/prbug/

The above data was captured using a performance visualizer program on our "Vaygrim's Chance" mod pack test server. We have noticed a memory leak that was first noticed back in v1.03 of my mod pack, which used Project Red v4.5.1.51 and continues now with the current version. I suspect it stems from the routing tables function of the Project Red piping system, unfortunately.

In the above data, click on "Details >>" towards the bottom of the page. This will take you to https://ace5852.com/prbug/pages/18.html which goes through a list of all things currently occupying memory that is being used by the server.

This server has 4GB of memory devoted to it, and usually only has 2 or 3 concurrent players on it.. on average. If you scroll down to the bottom of that page of data you'll notice the following line:

mrtjp.projectred.transportation.Router First 10 of 34,848 objects 34,848 2,230,272 3,726,632,456

Basically.. this one single item from Project Red is taking up 3.7 of the 4 GB of server memory. The problem starts off small but progressively grows over the course of an hour, forced the server into a crash. We have discovered that this issue can be mitigated using better Java Args however.. it only delays the problem by a few hours. It does not RESOLVE the problem.

Can you please look into this, MrTJP? I would greatly appreciate it. If you need access to our Test Server for the sake of performance evaluating, you are definitely welcome on it. Just let me know. Also you can always find me in Esper.net IRC under the channel #Vaygrim

MrTJP commented 9 years ago

Wow, that's weird. Approximately how many pipes do you have on the server?

On Nov 20, 2014, at 9:40 AM, Vaygrim notifications@github.com wrote:

Captured Data: https://ace5852.com/prbug/

The above data was captured using a performance visualizer program on our "Vaygrim's Chance" mod pack test server. We have noticed a memory leak that was first noticed back in v1.03 of my mod pack, which used Project Red v4.5.1.51 and continues now with the current version. I suspect it stems from the routing tables function of the Project Red piping system, unfortunately.

In the above data, click on "Details >>" towards the bottom of the page. This will take you to https://ace5852.com/prbug/pages/18.html which goes through a list of all things currently occupying memory that is being used by the server.

This server has 4GB of memory devoted to it, and usually only has 2 or 3 concurrent players on it.. on average. If you scroll down to the bottom of that page of data you'll notice the following line:

mrtjp.projectred.transportation.Router First 10 of 34,848 objects 34,848 2,230,272 3,726,632,456

Basically.. this one single item from Project Red is taking up 3.7 of the 4 GB of server memory. The problem starts off small but progressively grows over the course of an hour, forced the server into a crash. We have discovered that this issue can be mitigated using better Java Args however.. it only delays the problem by a few hours. It does not RESOLVE the problem.

Can you please look into this, MrTJP? I would greatly appreciate it. If you need access to our Test Server for the sake of performance evaluating, you are definitely welcome on it. Just let me know. Also you can always find me in Esper.net IRC under the channel #Vaygrim

— Reply to this email directly or view it on GitHub.

Vaygrim commented 9 years ago

Some rough math:

2 Routed Junction Pipes 15 Routed Interface Pipes 1 Routed Crafting Pipe 1 Routed Request Pipe 50 (roughly) Item Transport Pipes

There are only 2 'intersections' of piping, so I have a Routed Junction Pipe at each one of those intersections. There is only 1 chest that is being pulled from into the system, and only 1 Request Pipe being used to pull 'out' of the system.

Let me know if this isn't enough information for you. Also some of those numbers are slight guesses as the Test Server crashed overnight due to the memory leak, and I can't get ahold of the admin right now for a reboot. I can get more exact numbers later.

Vaygrim commented 9 years ago

It should also be noted.. I'm the only person on this server (that has roughly 10 players using it at various times) who actually uses the Project Red piping system. So I'm the only one with pipes.

octylFractal commented 9 years ago

I've got about 3-4 junctions, at least 70 item transport pipes, 4 interface pipes, and 7 crafting pipes. No problems at all.

Vaygrim commented 9 years ago

This issue has already become quite bad on our Test Server, but I am already seeing similar memory leak issues in my Single Player world, which also makes use of the Project Red piping system. The longer a recording session goes, the worse my game's performance becomes.. with minimum memory being used climbing higher and higher until it finally maxes out my client and the stuttering then becomes unbearable.

We have had several "Out of Memory" crash errors on the server before we tried to solve this problem by updating Project Red. I'll try to see if I can get those crash logs from our admin, and post them here.

In the mean time, this is a definite issue on our Test Server. Even if Project Red is not DIRECTLY at fault, it is definitely interacting with some other mod that's CAUSING this fault... and I'd really appreciate your help trying to track this down. If you want to do testing on your own, you can grab the "Vaygrim's Chance" mod pack, version 1.03, from any of the three major launchers. (FTB, Technic or ATL).

Vaygrim commented 9 years ago

I've contacted our server admin. Currently OOM crashes don't generate errors, so we don't have any logs on that front. Sorry!

octylFractal commented 9 years ago

Is it possible to debug the source of the buildup?

Vaygrim commented 9 years ago

@kenzierocks I have a silly question for you: Are any of your Project Red pipes hidden by FMP covers?

octylFractal commented 9 years ago

No. I can add some.

Vaygrim commented 9 years ago

@kenzierocks In all of my Project Red piping setups.. I've got the pipes covered with Forge MultiPart covers. I'm wondering if this is what is having an impact / interaction with PR to cause memory leaks. (This is literally an uneducated stab in the dark.)

MrTJP commented 9 years ago

Covers are probably not the problem. Code wise, covers literally just sit there. Ill keep investigating..

Vaygrim commented 9 years ago

So we have gone ahead with an experiment on our testing server. We completely removed the Project Red "Mechanical" jar file, and restarted the server. It has not successfully run more than 12 hours straight with zero sign of a memory leak.. something not possible when PR_Mechanical was installed. We will do further testing but right now it is looking like the removal of Project Red Mechanical actually SOLVES the memory leak.

Does this help narrow down the issue at all?

MrTJP commented 7 years ago

I am closing this issue, as there isn't any concrete evidence as to how this memory leak occurred, where it is, or how to solve it. I have yet to run into this issue.