Possible client memory leak with large number of active fuel cores.

[raelik]

So I've noticed an issue when running a large number of fuel cores (96 in my case). I have 8 separate reactor setups that look like this:

(except that the right 3 neutron reflectors that you can JUST see at the top have been replaced with bedrock ingot blocks)

Each of these sets of 24 cores (3x3 donuts, 3 levels high) only have reflectors on two sides, with one side totally unshielded (facing another set of cores nearby), and the other side mostly shielded with bedrock ingot blocks (facing a different set of cores much further away). At lower operating temps, this doesn't seem to be an issue, or with not all of the reactors running full-tilt at the same time (I've had 6 going with no noticeable issue). When I run all 8 of them (so the full 96 cores) going full-blast though, with the average operating temp around 250-300 degrees, my client grinds to a halt, assuming that I'm in the same dimension nearby. First I get rubber-banding and severe interaction latency, then the FPS stars dropping and finally I get the "out of memory" screen:

This is using the latest releases of DragonAPI, RotaryCraft, ReactorCraft, and ElectriCraft.

[ReikaKalseki]

It might just be entity overload. Start the process again, and before it crashes, run /selectivekill EntityNeutron 100. How many does it clear?

[raelik]

If it does the same thing, I did try "/cofh killall neutron", and it killed ~850 of them (I think 847 was the exact number).

[ReikaKalseki]

850 is not very many, nowhere near enough to cause lag.

[raelik]

Could it have to do with the fission cores chunk loading? I'm already chunkloading using FTB Utilities.

[raelik]

I tried disabling visible neutrons, which didn't help at all, more or less proving that wasn't the issue.

[OvermindDL1]

I tried disabling visible neutrons, which didn't help at all, more or less proving that wasn't the issue.

Well if you are getting rubber-banding, that sounds like the server-side is being overloaded (which in single player the server is embededed in). I'd hook up a java profiler at that time and see what's happening for a few second chunk of recording. If you could post that profiler snapshot then we could take a look.

[raelik]

@OvermindDL1 It's definitely not a server-side issue. If I leave the dimension my reactor is in (a Mystcraft age), and go back to the Overworld, there is no noticeable affect. Tick rate is fine, CPU and memory load don't increase in any noticeable way. I should be clear that I'm running a dedicated server on a completely separate machine.

[OvermindDL1]

And yet you get rubber banding?! Only other thing that would cause that I'd think then is if there are too many packets being sent....

[raelik]

That was my guess too, though why that would cause the client to die with the out-of-memory message, I dunno.

[OvermindDL1]

Some thoughts:

• If the client isn't processing them fast enough.
• If the packets are creating deferred actions.
• If the packets are creating 'things' like entities (the commands only wipe server entities, not client-only entities).
• If a mod has a memory leak.

Etc...

A profile snapshot would help a lot. :-)

[raelik]

I'll see what I can do. I've traditionally had issues getting a profiler to connect to minecraft.

[OvermindDL1]

What issues have those been? It's been quite reliable for me to date.

[raelik]

It's because I'm running a modpack through Twitch, so I've had issues getting it to properly start with the profiler connected. Not gonna lie though, I just don't know enough of what I'm doing there.

[raelik]

If the /debug command works in 1.7.10, I could just use that.

[OvermindDL1]

You don't actually need to start with the profiler connected. Just start up the game, get it close to where the issue happens, launch the profiler, 'Attach' it to the MC process, start recording snapshot and then quickly cause the issue, then when it crashes save out the snapshot and zip and send it over. :-)

No clue what mod adds a /debug command, so no clue?

[raelik]

I think my issue is that I've always tried to use my IDE's profiler, which most of them don't have that feature of attaching to a running JVM (IntelliJ doesn't anyway). I'll find one that does and get back to you.

[OvermindDL1]

jvisualvm is the one that comes with (at least my installation of) JDK, the usual java profiler. Really old swing-interface but very functional. :-)

[raelik]

So yeah, can't use jvisualvm with Minecraft (not with this modpack anyway). It hoses the client when you try to start the profiler. CPU causes it to get a network timeout, and Memory just completely locks it up. Also tried WarmRoast, that just flat out crashed my client when I tried to attach :P

[OvermindDL1]

Heh, how much RAM do you have and how many cores do you have? Assuming sufficient amount of both (I have 6 native cores with 16 gigs ram and it has no issues here, though I do use linux) then it shouldn't freeze, even while profiling.

It might be time to try making a minimal reproduceable example with a set of instructions for us to perform in that save and modpack to cause it?

[raelik]

I'm on a devil's canyon i5 (so 4 cores) with 16G of ram, so similar. So if you wanna do a test, here's a layout that should replicate what I built:

The green X's are fuel cores, blue squares are boilers, red \'s are neutron reflectors and black X's are bedrock ingot blocks. The purple areas are plasma injectors in my setup, but just putting stone or something else in there may also work, not sure how those injectors interact with neutrons. All of these are 3 blocks high.

[raelik]

If it doesn't cause issues on its own in that configuration, it may be a side-effect of being operated in close proximity to (i.e. literally built around) a fusion reactor :P

[OvermindDL1]

I can't actually build something like that myself, the renderer of Reika's stuff is extremely inefficient and I have an older card that doesn't autobatch so I get around (or less than) 3fps with Reika's mods when too many renderables from it exist (60fps normally sans reika's mods or anything from it existing)... ^.^;

Something pre-built and ready-to-break I could though as 3fps (or less) is fine for that. ^.^;

Still though, 16gigs ram with that chip, shouldn't really cause the game to pause, how odd...

[raelik]

To be fair, none of those blocks animate, so if you turn off neutron rendering on the client, it shouldn't have any GL impact at all. I tried turning off chunk loading for active fusion cores as well (didn't help, which I didn't think it would), but I did notice a particular behavior. The rubber banding gets worse the longer I stay near the running cores. If I teleport back to the overworld and then come back, it "resets".

[raelik]

Just looking over the various bits of relevant code (TileEntityFuelRod, TileEntityNuclearCore, TileEntityReactorBase, TileEntityNuclearBoiler), I can't see anything that would be responsible for this, at least, not on its own. Wondering if it might be an issue with the Ender IO conduits I'm using to add fuel/remove waste. That said, the effect doesn't happen until the reactors start producing neutrons, and it doesn't stop if I shut off the flow of fuel... though it could just as easily be from all the waste being produced. I've used Ender IO item conduits liberally before without effect, but I don't think I've ever had so many item conduits in one area. I'll try switching to something else.

[ReikaKalseki]

You can also try using /debugtest, which will constantly delete all waste and spawn fresh fuel as necessary. But you must back up your world before doing this, as it will leave data on a variety of blocks that puts them into a sort of pseudo creative mode.

[raelik]

That's probably worth a shot. That way I don't have to break any of the conduits if it turns out they aren't actually at fault.

[raelik]

So after removing the Ender IO conduits and using /debugtest, the issue is gone. I'll go ahead and close this, since I really don't think the fuel cores have anything to do with what I was seeing.

[raelik]

So I closed this too soon. I tried with TE opaque warp itemducts, and ran into the same issue when I fired all of them up at the same time. There must be something going on with the fuel core inventory handling

[raelik]

Also, there's 192 fuel cores, not 96. Dunno why I thought that.

[ReikaKalseki]

The item insertion code is very simple...

[raelik]

Yeah, I saw that, but the only time this starts becoming a problem is when the cores have to actually have fuel fed in and have waste removed. The only thing I can think to do at this point is remove the connecting itemducts, load up on fuel, and just manually load them up, letting the waste just build (doing a backup beforehand). That'll at least tell us if the issue is really internal to the cores, or if it's an interaction issue.

[ReikaKalseki]

Go ahead and try.

[ReikaKalseki]

So...

[raelik]

Sorry, real life got in the way of me testing this, haven't had the opportunity to try that last thing I mentioned.

[MachiavelliSeraphim]

Im fairly certain TE item ducts shouldnt cause lag at that load level.

I had a TE item duct system transporting fuel and waste in my reactor and they were easily holding their own against 128 fuel cores and 160 breeder cores.
Fuel was being supplied and waste removed by ME system Interfaces and transported by Resonant Extractors through impulse itemducts.

I was also using super laminar ducts with resonant extractors to deliver coolant and remove hot coolant via tesseract.

Definitely worth a try removing the fueling system. Another thing Ive done in testing is to draw the waste from the cores with a resonant extractor and impulse itemduct and dump it straight into a void chest.

If the waste isnt removed, the reaction slows and stalls and doesnt really give a good stress test in my experience.

Last suggestion is to try Impulse Itemducts instead of warp. I have seen higher loads when using warp itemducts.