Sub-networks causing huge performance issues.

[joshricker]

Going to put this in a new issue as this seems to be far more broad than I thought, affecting everything. Using subnetworks starts lagging the server, and depending on how many, it can be very extreme. May be related to #127 and is certainly linked to #676 but not only affects crafting but server performance as a whole. Heres the link to a test world I used to find this issue using only rv2.beta build 10 and forge 10.13.2.1291. http://puu.sh/gzmhA/0fa96eaf24.zip If you just load into the world you will immediately notice it, just try crafting something, or even just opening the terminal. For some reason /forge tps will not pick this up. Opening the world in an instance with opis show just how bad this is http://puu.sh/gzlWP/4fdbb8ed8d.png, this is on a local game.

[yueh]

This is not related at all. It is more about trying to actively break AE2 by introducing (unnecessary) loops. We already take care about infinite loops and prevent them, so it will not crash your server. But without any hints the network needs to bascially fully traverse every possible path as each has the same priority and this can potentially take some time.

There is a reason the priorities of parts like the storage bus exist. Just give the network some hints about the best possible path to take and the issues will be mostly gone.

But we can only prevent issues to a certain point. If a user for example decides to drop many stacks of cobble to impact the server, you could decrease the despawn timer. But that just means drop more items and do it faster, until you decide to turn off dropping items completely. Which is not really a solution.

[Cisien]

I'm curious if the pathfinding work can be made asynchronous and offloaded to another thread?

[yueh]

The pathfinding does not care about any other network. Just assign channels to the local one.

This is just about where to find the inventory containing some items and if you let it decide to let the network handle it, it will potentially take the longest path and loop around as often as possible. Also done for each operation, as just know the item is behind that storage bus. But than can be 1024 additional hops to take or just one. The requester has no information about it.

[joshricker]

I understand that the save is overkill as it was just to show it in its worst state. If you look in #676 you can see why i started testing this. I wasn't using 20 repeaters networks, or even 5 of them. Only 3 of them and was able to bring the server to a halt for mins at a time. The one thing that is not in the save is a real base, some form of network complexity. I would be more than happy to get you on the server I run and show you first hand how the crazy snapshots i've showed you are made.

[yueh]

The save is in no way overkill. It is just mostly a poorly chosen example.

First you introduce dual way connections into the system, which results in each subnetwork being able to access the items through one of the 2 potential paths. Worst case it will traverse the longest path. (I am currently suspecting it either choose the nearest or farthest away storage bus from the controller as first option). But that overhead is completely negligible. I have a similar setup with about 10 loops and no priorities. But it has absolutely no performance impact. It is even that small, that it is not always picked up by the visualvm sampler.

Secondly you network has a way more important flaw as the furnaces are not setup correctly. They are constantly exporting a single piece of coal and immediately import it back. This is the actual problem. AE2 is mostly designed to work without being affected by the stacksize, so there is not difference between moving 1 item (of the same type) around or 2 millions. But that comes at a cost for handling a single itemstack and constantly moving many itemstacks with a stacksize of 1 around will have some impact. It also needs to propagate every change of a single item through all connected networks, so the network design will affect it a bit. But this is a minor portion of the overall work.

Yes, abusing it might be not that difficult. But dropping a few thousand stacks on the ground is also not difficult and will also have an huge impact on the server. Or any other mod interaction bug, like there are still some tileentities left, which can corrupt a world save when moved with frames.

[joshricker]

Is there just no safe way to use repeating networks? You talked about priorities, would setting them on my subnetworks help? And if so how would you set that up? As much as I love using the storage bus to interface I would suggest removing that interaction. Maybe there is underlying bug or improvements that can be made but it seem just way too easy to bring down a server with it. And about the coal spam that was intended in order to make tons of traffic. With no subnets the system made no lag whatsoever. Here is a save with the furnaces connected directly into the main network, with the coal spam still happening. http://puu.sh/gAUMW/a4fe1e5145.zip I saw you say that the coal spam was only a small part, but what is causing the lagest part? Is there a way to use subnetworks, that can be recursive but not create loops? It seems unavoidable given how they seem to work but I would rather disable the interaction and just force my users to use P2P. As just one subnet is noticeable, more than that and the network is unusable along with impacting the server as a whole. And with the stack dropping reference we as server owners know there will always be ways to crash a server, but we do what we can to help fix the issues and make sure they are known. As the part that seems the worst is its invisible in opis and every way to check tps I've tried. So if this is happening on your server all you know is its AE somewhere with your only option to roll back to when it wasn't lagging. At Least with drops you know right where they are and have the tools to fix it, even bad tileentity isn't that much for a server owner with NBT Explorer.

[g1xb17]

I'm curious, What is it that is actually bad to do here? Storage buses to the same network to get a 2-way network? Then exporting and importing back and forth, so that it doesn't have a clue which network to use?

[yueh]

@gixbit mostly that. It technically introduces an infinite loop/recursion into the network. AE2 itself is intelligent enough to avoid crashing with an stackoverflow or similar. Because it can detect an already visited network.

But like any algorithm it follows some basic rules. For example you want to explore a labyrinth. The most simple way is to say always turn left. It might take ages to explore it, but it will succeed as long as there are now loops. So to avoid loops we just mark our way somehow and just turn left again once we hit an already visitied path. So even with loops we will succeed sometime. If the person designing the labyrinth knows these rules, the can just place the exit directly to the right side of the entrance, resulting in us only finding it after it is fully explored. Then making it just longer will take more time. We could of course be a bit more intelligent and just look right for an exit before turning left. But we will still miss the exit, if it is just a corner away. As long as we are following rules, there will be always one or more layouts for the worst case. Exploring it will always take the maximum amount of time. But at least we can easily guarantee to solve it and not die, because we got stuck in a loop.

But there is no real reason to be mean and force the worst setup on AE2. Instead you can use the tools we give you (like priorities and filters) to tremendously improve how it explores. Basically saying "follow that line for the quickest path".

So removing that interaction is just stupid. It is exactly the same as remove the ability to drop blocks, because you can crash the server with it. You will probably do it once or twice and crash the server. But you can learn to prevent it. It is an issue, which you can solve by yourself (like not decide to drop 5 million cobble) and not something unpreventable (like each time a player joins the server, some mod drops 5 million cobble). But of course we can try to improve it in general, but as long as you can throw more stuff at us, it will always happen. Just not so fast.

P2P tunnels for example have way more issues and these cannot be solved by the players themselves (except switching to better designed subnetworks). So if we would discuss about anything to be removed, then P2P tunnels are having a way higher priority for it.

[g1xb17]

Yes, reminds me of tail recursion in a maze type scenario. Doesn't matter if you go left or right continuously, the result is the same and can still take variable amounts of time depending on the number of pathways.

I would hope you do not remove tunnels, they're very useful. I think my only question is, If you haven't used priorities or filters on storage buses, is it okay to not do this at all, as long as you don't set up dual networks? I personally prefer 1way sub-networks with a master network.

[Blue64]

Very informative, I hadn't even thought about a recursive network yet, though I have toyed with the idea of a master network fed by multiple other networks, the master network will of course be my main storage and crafting. Based on what I've read, if I understand correctly, a 1-Way connection should experience much less problem with this issue, as there's fewer loops. Also I believe I may have located the source of my Lag when viewing my full ME Drive, it's doing a recursive loop because I've got them on a 2×4×1 layout on a single Cable running along the 4 side, if I line em up, with no recursive connections, I'm guessing it's going to run much smoother.

[g1xb17]

We're not talking about cable here. We're talking about P2P networks when you connect a P2P to one network and then connect another one back to the first network. Or, In string of networks such as. Where it has a property such as.. 1 > 2 > 3 > 2 > 1 > 2 > 3 ... or ... 1 <=> 2 <=> 3 which shows the possible ways to travel. You can see where there lacks an end point.

Your drives should be fine. When things are on the same cable, it uses convenient pathfinding and generally doesn't physically change, unless, you add more devices that use channels on that cable. Also, there's a known issue when browsing the terminals and clicking stuff out of the terminal causing some freezing type issue. They're saying something like it might be fixed on rv3 bleeding edge right now or something. But, I dont know if thats true myself.

The only reasons I can think of that someone would want to set up a recursively configured network where everything is doubly connected or greater, is to provide the ability to see the same items in both networks which is a design flaw on the players part. What this means is they do not have the ability to think about in terms of the master -> slave configurations and they're provoking the non-issue argument for AE2 Devs. You can't have both. And whether you set up storage buses from a "main network" so that you can see items from the main network and another network or vice versa. The proper design is that you will always end up with something like .. 5 < 3 < 2 > 1 > 4 < 6 < 7, Where nothing is recursive and that an endpoint can be found when browsing from Network 2 and Network 7 cannot reach Network 2, WHICH I would argue is the main network. IT MIGHT be okay to connect from 7 to 2 directly, but it would need to be tested to see.

So if you want to do things from the same network, providing items to other places, it's got to be a challenge you have to overcome, like quantum rings and p2ps from the main network. You have to end up making bigger controllers to support more tunnels, and then you get to a certain point where the main network just consists of two controller structure where the only things it's used for is channels and the other controller to tunnel channels to the place you want to connect. You can still use other networks but, if you want the main one to be used in certain cases, you need to do it so you can properly incorporate it.

[Blue64]

With that layout, you'd be able to safely connect 7 to 2, but you'd get minor lag if you grabbed from 4 via 7 as it's a double check. Initially they were discussing Cables, the P2P was mentioned later because it helped cut down on lag spikes.

When building any network, like with any structure, whether physical or code, it's only as strong as it's foundation. If you want a master network, then start by building that, and expand from there.

If you want a fully recursive network, you'd better have an Alienware Computer or better (think Home Server) to back up all that power you'd need to do that math.

[yueh]

Please stop necroing issues, especially if you just post plain wrong information to it. I really do not want to start needing to delete posts or even lock issues down, as we are the ones having to deal with the rumors about "AE being slow, because I heard a friend of a friend talking about a random guy in a random forum claiming X causes horrible issues"

[g1xb17]

I did not say that without any information yueh, I also experienced this on the FTB infinity pack. They're not using rv3 yet. It's unlikely they will until rv3 stable. No rumors here. Just cold hard facts from another issue.

https://github.com/AppliedEnergistics/Applied-Energistics-2/issues/1448

I should have said that there is a known issue with shift clicking itemstacks. But, that it is likely fixed in rv3. Because that is the one issue that should be most noticeable to almost everyone who uses AE. Honestly, For what AE does do, you guys have done an excellent job at making it work efficiently. Anyone who understands a bit about data structures and programming can respect that. It is quite literally the most usable mod you can get.

[Blue64]

As far as my information goes, I've been doing lots of reading, mainly of Yueh's posts across a wide variety of similar topics. The information about the double check on the example above was because of a post where Yueh explained in great detail how the AE2 system is built to prevent a server crash on infinite loops by either always going left or right and marking any paths it has already taken (just in case) so it doesn't get lost. That's where this issue originated from, and as Yueh explained, the priorities feature helps prevent it further from getting lost in an endless loop, along with filters. The part about the P2P reducing the Lag Spikes was true because I actually ended up here from it's original post, which is referenced in the initial post here. I do agree that it does seem to be everywhere Yueh, but as we all know, bugs only really need to be reported if you're actually using the latest version (in this case it's the latest version of rv2, and it's unavoidable because the problem is fundamental and not programmatic, y'all did your best, but someone, somewhere, eventually, will attempt to break it, that's the entire idea behind so many things, nobody likes people who purposefully throw down 10k stacks of cobble to crash the server, but some people still do it for whatever reason, but having safeguards in place, such as the Stackie mod on Servers, can help prevent crashing), a large majority of these posts would vanish if they knew what version they had and could compare it to the latest version, which is why that's always your first question, along with requesting the crash log. You're doing a very very good job Yueh, along with the rest of the team, and I do hope this fundamental problem one day isn't a problem, but I won't hold my breath for it.