TPS lag coming from AE2

[Eufranio]

Recently my server started to drop TPS, I did some profilings and i got one: Report.zip Could you help me fixing this? Play with 5TPS is not good :/ Using AE2 rv3-beta-6 Forge 1614 Minecraft 1.7.10

[Meller127]

Just a question, do you have any Deep Storage Units (DSU) connected to your network?

[yueh]

Caused by missing chunkloaders and/or ones, which cannot keep a chunk actually loaded and force them to constantly be unloaded.

Somewhat similar to #2338 and probably other reports. But the first time someone also added a sampling dump. Which points to ae2stuff being a massive part of the performance problem. With about 1/4-1/5 of the total CPU time taken by it. The rest is mostly taken by core java classes, we might be able to streamline their usage it a bit.

It looks like they are even affected when other parts of the network gets unloaded. No idea about how many of them are around. But if that is just one or two in a single network, I do not even want to think about what happens, should someone scale it a bit more aggressive.

[Eufranio]

If you still developing AE2, could you add some debug code to help me finding the problematic network? :/

[yueh]

It's not really AE2, but some chunkloader going haywire. Should they not provide tools to debug them you can try /ae2 Chunklogger, which enables/disable the logging of every chunk being loaded or unloaded to the console. That might be enough to find the chunk.

I have two changes for AE2 in mind. But one might throw CMEs and no idea how the other one would affect the performance. But neither will not be backported to 1.7.10

[Eufranio]

God, why not? D: Well, I use chunkloaders from MineFactory Reloaded, should I switch then?

[Eufranio]

Report2.zip I've removed the chunk loaders, and there's no output after enable chunk logging... The TPS now is 4, and removing AE2 fixes the problem but I need it, It's so usefull and everyone loves it =/

[shartte]

@Eufranio If there's no output after enabling chunk logging, something is broken. The output is on your server log btw. I can see chunk unloads in the profile report you provided, so you should see output. You can also enable additional tracing in the config (ChunkLoggerTrace), but you'd still have to enable it using the command that yueh posted. The output in the log should look like this: Chunk Loaded: x,y

[Eufranio]

Server log, not console, then? I'll try, be right back

[shartte]

I checked the command implementation and it's doing this:

AELog.info( "Chunk Loaded: " + event.getChunk().xPosition + ", " + event.getChunk().zPosition );

So I'd assume server log.

[yueh]

Also the last snapshot makes no sense. Like it spends about 10% in a method done nothing but return size > 0 with 48 invocations. Similar to the massive timesink HashMap.putVal() is with 147 invocations. There is something way off.

Also the usual stuff about "do not use bukkit", we have no idea about what it screws up.

[Eufranio]

Sorry, this last report was a profiler of only 1 second, something was wrong. I've profiled one more time, by 1 minute, here is the report: Report3.zip

I've also disabled all the chunk loaders, still with TPS 3 now... :X

[shartte]

@Eufranio We've looked into it a bit, and there's not much conclusive evidence of what's actually wrong in your case. We've discussed if something in your huge lists of mods/plugins may interfere with the HashSet, but again, no conclusive evidence.

[mindforger]

@yueh

Also the usual stuff about "do not use bukkit", we have no idea about what it screws up.

bukkit implements an event system, maybe whenever a plugin-listener is triggered it uses some sort of "get" mechanism making all your code run again and again, multiple times, every tick ?!

[Eufranio]

Okay, I've took a look in the Bukkit config, and it unloads chunks, accidentally i've decresded the ticks between unloads, sorry =X

But now i'm having a other lag problem, not related to AE2 maybe, but could you guys take a look? I'm new reading report files :/ Here's the report:

Report4.zip

[shartte]

Okay, I've took a look in the Bukkit config, and it unloads chunks, accidentally i've decresded the ticks between unloads, sorry =X

You do realize that this reinforces yueh's opinion about supporting Bukkit servers, right? :-P

Looking at your performance report, it does not look like AE is taking up a significant amount of time on your server anymore. The only thing that catches my eye are the insane number of LUA threads you have. How much OpenComputers / ComputerCraft are you running on your server? It looks insane.

[Eufranio]

Yes, bukkit seems to be the worst thing to the modders :V Well, I run a public server, i don't know how much computers are running... Is this causing lag? Because I saw they were in a different thread, so i thought was not a heavy performance issue

[shartte]

is this causing lag?

Do not take my word for it.

[Eufranio]

According with my Opis results, CC really seems to be consuming a lot of the tick time...

[mindforger]

computercraft is one of the most (in multiplayer) abused mods imho, you can reduce disk size of computers in the config to prevent monster scripts running but it only takes 3 to 5 lines to bring down the server TPS

[Eufranio]

Okay, fixed the chunk loading problem, but with the chunks loaded AE2 seems to be lagging a lot more, see: WarmRoastReport.zip

Is it same as #558 ?