Unplayable Lag with ME System & lots of learnt items

[ghost]

Using v0.4.6 in Project Ozone 3, at least a thousand items learnt.

I've been playing for a while and the lag's been getting worse and worse till I'm getting 10fps at 5 render distance and I can't play anymore. Lag goggles show EI is using 190% for the server tick event. I've tried limiting it to 1,000 items shown (from the default 1,000,000), but it hasn't done anything.

When I tried opening my terminal while timing, it took several seconds for any items to appear, and then the whole game froze for 30 seconds (windows reported not responding).

latest.txt

[ghost]

I just did some extra testing and it appears that it has something to do with AE trying to get a list of all the items. If I set the storage bus to insert only, the lag goes away and extract only/bi-directional make the game laggy. Disconnecting the transmutation chamber also reduces the lag.

[random53]

Im curious of the specifics of this because i created a dummy me system and the lag was at least reduced by a significant amount for me. Ill have to do more testing to see whether it could be based on just the sole combination of items, autocrafting, etc

[random53]

Also seems like the mod author hasnt really been around much in the past year about so im doubting for a fix to happen anytime soon

[ghost]

If you use a tome of knowledge, does it get any laggier?

[random53]

I only did survival testing so i didnt have a tome of knowledge unlocked yet. However after spending a few hours messing around with the system im pretty sure export busses and import busses cause 90% of the lag.

[random53]

After replacing all of the export busses that i could find and setting the storage bus priority to negative ( so items didnt get imported into the transmutation chamber) the impact went from 36ish% to 1% and im pretty sure i didnt get all of the import busses but either way its a huge difference and doesnt really even seem noticeable

[random53]

There could still be other stuff at play but when i was testing it with only drives hooked up it was down to .27% at most so im assuming the readout updates arent the problem just the emc conversion updates

[ghost]

Another edit, it appears that it's most likely due to me constantly exporting items from my ME system (using the transmutation chamber). I have 3 quantum compressors with export buses on them, each using an EMC'ed item. As soon as I added a toggle bus and disabled them, the lag immediately reduced. My best guess is that each time AE exports an item, it checks how many of that item there are, which would call an EI function. I saw in another thread that it's a limitation with ProjectE, where the function to get an EMC value is super slow.

[random53]

Yeah something like that it could also be the fact that the system doesnt actually have those items stored so the export bus probably has a wierd functionality issue with the emc pool

[random53]

Interfaces seem to work just fine as far as i can tell so just replace them with that and pop some translocators on

[ghost]

For the compressors I just used automation to get all the catalysts, components etc. I know for a fact that the export buses were pulling from my EMC because I don't think anyone has millions of blocks of ludicrite to compress. And yeah I switched to Personal EMC links and those were fine. I think it's just an issue of AE exporting an itemstack, then marking the inventory (the transmutation chamber) as modified and tries to refresh all the items stored in it. When you have several thousand items stored, calculating the EMC and amount of items left is going to become pretty slow. I saw somewhere that ProjecE takes over 6 seconds for 100k EMC-able dummy items. So ~60ms for me.

[random53]

Well i asked before i saw you only said you had 3, but i was wondering because they are animatable but i feel like that seems kinda cheaty? but idk

[ghost]

Animating is the entire point of the pack. It was purposefully left enabled. Anyway I'm very late game (almost up to infinity gear) so I don't need to duplicate that much stuff.

[mkaito]

The big lag source isn't just that there's a lot of different items, but that the transmutation chamber tells AE2 that you "have" 1 million of each item. That's a whole lotta stacks.

You might want to try changing the setting for max published items. I reduced it from 1m to 500k and the lag is considerably better. The only downside is that you can't queue massive autocrafts that actually need 1m of any given input, but I haven't had that problem (yet).

As you guys found, bulk export/import via transmutation chamber is quite laggy and EMC links work better in this regard, especially when bypassing AE2 entirely.

In general, AE2 performs really badly when you bulk-move items by digitizing them and then immediately un-digitizing them, such as importing via storage bus only to dump it via export bus. This is one of the Two Big No-No of AE2 performance. The other one being a storage bus loop (interface stuck on a storage bus on a subnet, and again the other way, which results in storage data updates looping exponentially whenever an item is moved via either storage bus).

This isn't just specific to the transmutation chamber, it's also pretty bad if you start exporting at full speed from a black hole unit, for example. Exporting from ME Drives is considerably faster.

Bulk exporting from the transmutation chamber just adds more lag on top of an already laggy operation.

Anyway I'm very late game (almost up to infinity gear) so I don't need to duplicate that much stuff.

Boy oh boy, do you have a storm coming :joy:

[random53]

I remember from my testing that when i changed the max items down to even 1000 the lag still largely remained. The only thing that fixed the problem for me was allowing no imports into the chamber itself and only either exporting with the chamber disconnected or in small amounts.

[2000game]

I have a similar issue, while not having inserted the tome of knowledge all works fine, but when I insert the tome of knowledge the server crashes at some point from being behind in ticks.

I found this issue, which is the same one, thought he author claims that he fixed it in 0.3.0, Project Ozone 3 currently runs 0.4.6 which is the newest version

I changed I:maximumExposedStackSize from 1.000.000 to 100.000 , I nearly thought the server wasn't crashing anymore and wanted to write that that worked but he crashed again, though it took longer than before.

[jk1895]

Please fix this bug.

[MarkRyanDev]

I'm playing Project Ozone 3, and once I learned the tome of knowledge my tick rate tanked from 19 to 3. I removed my AE2 storage bus as recommended by another user, but it was still really bad. Then I tracked down my other transmutation chamber, which had a single item translocator on it (not even connected to anything!). When I destroyed the translocator my tick rate went back to normal, even with the storage bus reconnected!

[Zhaocl1997]

Playing po3 mythic mode, still got the problem. My tps is totally ok but the world time in the screen's left go back sometime. I think the problem cannot be solved since author not matintain this anymore so i just shared my experience.

NO AE2 INTERFACE/EXPORT BUS FOR EMC'S ITEMS JUST USE PERSONAL EMC LINK OR THE COMPRESSED ONE TO EXPORT INFINITE EMC'S ITEMS

I have tested in my own surival saves with laggoggles, the result is very clear.

Here are the screenshots.

I got a set of machines to generate all the wafer for chaos catalyst. There are 32 ae2 interfaces exporting the input materials.

when these 32 interfaces is connected, the result with laggoggles is like 9w+.

when i disconnect the 32 interfaces, the result is like 5w+.

So the result is clear, even if using the interfaces to export emc's items, it's still lag the world for many perspectives.

I'm gonna replace all the interface and export bus which exports emc's item with compressed emc personal link.

I'll post anthor screenshot when the reconstruct is done.

[Zhaocl1997]

While, after the change, I found that emc personal link also lag the world.

Below I put like 40 emc personal links and the laggoggles result show that emc personal link lags more the ae2.

Then I decide to use one emc personal link and distribute the items with xnet, the lag reduces a lot in personal link.

Consclusion: No more ae2 export bus/interface to export emc's items or too many personal emc link. The world is so much better now!