[1.10.2] Items getting "stuck" within pipes

[NeOAnderson2005]

Hello so I have a fairly large pipe network over my base which spans i'd say 50 blocks wide, maybe 100-150 blocks deep. This network pretty much is self-managed in all aspects. By this I mean, simple things like ores from quarries coming into the storage system, (when available) , will be requested by pulverizors and broken down into dust which is then sent again via the pipes to smelteries when available and then back into the storage system where it will sit until the PR Auto Crafting Bench will request and craft the ingots into blocks to save space. This is just one part of the system, along the same network I have various access/request pipes over it and various other systems like the mob grinder sending items into storage e.t.c. Now whether this is the issue or not, that it is simply too large ..? i'm not sure but as time has gone on especially after updating to the FTB Beyond 1.8.1 Pack (if that helps) I have noticed that at specific points within the network (usually where there's a routed junction or Routered Interface pipe) it will get "stuck" in the sense that it doesn't pass that point and gets sent back to along the pipe whereby it will then get stuck by the next Junction Pipe or Interface Pipe and then switch directions again and effectively going back and fourth between the same section of pipe. I have seen various videos that explain having pipes in a 'loop' of any kind can confuse the items in the pipe and cause this same thing to occur but I simply dont have any loops. Especially between the exit of Pulverizor and the Alloy Smelters. It's a direct pipe going from exit to chest.

If you see from the screenshot, where my cursor is pointing is exactly where the items refuse to pass and get stuck in a back and fourth game until I break the pipe and replace it. Then it's fine.

[NeOAnderson2005]

Just like the above, I will show a few screenshots of other points that it gets stuck at and starts looping back and fourth:

This one is a bit strange, it will get to this junction point and most items (if not all) will go normal through the system to its indended direction however slime balls will slow down and change to a red aura around them as if there's no space in the chests on the network but there is. Infact, there is multiple chests with space and I even have a reserve crystal chest that is on low priority so when all the other diamond chests are full, it will use the reserved one as a final option (which i can easily see due to items appearing through the chests)

[MrTJP]

Ok, so it looks like pipes are loosing connection. If an item turns red it means the pipe knows where the item should go but can't decide how to get it there. Usually breaking and replacing the pipe should fix things like this. If you could recreate this system in a flat world with only PR installed it might help narrow down the problem

[NeOAnderson2005]

Hello there, thank you for responding so quickly. I have been conducting further tests at home and seeing what works and when things break , what corrects the issue to help diagnose the cause. It is pretty muche exactly as you said, it "forgets" that pipes are physically mapped and there. Yes you're quite right, breaking a pipe and replacing it, does indeed fix the issue temporarly. If you look at that last screenshot I have there, those two auto crafting benches I have there are sitting with 1 broadcaster and 1 stock chip which requests ingots and crafts them into blocks. Now I have noticed multiple times that it will (despite having perfectly enough power e.t.c) they will simply not request/get the ingots as if it cannot see the ingots on the network (hence earlier about losing connection) if I were to break that blue junction pipe and replace it. The ingots will suddenly flood out of the chests (which are down that corridor in the screenshot) and flood into the autocrafters at which point it does its intended job. Again, I dont really know why it keeps "losing connection" as such and I think this is the problem. I'm not really sure how I could re-create my exact system again without the other blocks however hmm. Could this have anything to do with being in different chunks?

[MrTJP]

Chunks containing pipes should be chunk loaded. I've designed the system to automatically re-link if a chunk is unloaded and reloaded, but thats not exactly 'tested' completely. How many routed pipes would you say are in your network? Also, in terms of recreating it, the actual blocks the pipes are connected to are not important, they can be replaced with chests or any other inventory, the pipes don't care about what type of inventory it is.

[NeOAnderson2005]

I assumed they would require chunk loading and we do, we use the built in claim/force chunk loading system with FTB Beyond 1.8.1 so that's set covering the whole base and pipes. I mentioned the claims simply from the point of view that perhaps crossing achunk is what is causing the issue but i guess not? providing chunks are loaded its fine. I would say if I were to put a number on it, easily a 100 interface pipes, not sure on junctions, maybe 30?, and this spans over a huge land with mulitple floors so re-creating it will be quite....a challenge XD especially as a lot of it is hidden within walls and such. Perhaps I can download the world and remove all non-pipe blocks somehow?? or perhaps I can send you the world? somehow?

[MrTJP]

So i guess the easiest way for me to diagnose this problem is if i have your world as well as a list of relevant mods so I can try and run it in my dev environment. Could you send that to me?

[Psy-Virus]

Hey @NeOAnderson2005 and @MrTJP,

Just a question: We're talking about the following, right?

Items get stuck - Youtube Video

If so, it happened to me either. @MrTJP, did you already get the world from him? If not, tell me and you'll get a full set up dedicated server folder incl. mods, so you can hopefully narrow it down.

If there's something else that can be done, I'm all ears.

:-)

[Psy-Virus]

Sorry to bother you @MrTJP, but were you able to reproduce the issue? Do you still need a world for diagnostic purposes?

[MrTJP]

I couldn’t reproduce it but I didn’t really have much to go on. I have plans to redo some of the routing code so hopefully that will fix it. This will be much later though. You’re welcome to leave a world download if you would like but I can’t promise I’ll get to it any time soon.

[Psy-Virus]

Hey @MrTJP, Thanks for the quick response!

Here you have a google drive link to the world (Server and client mods and configs)

I don't know how you will try to reproduce it or what techniques you'll use. But for information: The network is quite large and has two storage networks (like server farms) attached to it. The bug mostly occurs a while after the server was started when you request something it won't arrive and if you refresh the view, the item you have requested magically increases the count.

If you need any more details, please let me know, I am willing to help to get rid of this; first of all because my clients annoy me with this and because Project Red is simply the best, if not the only, mod to build community-storage-networks. Is there any debug-log stuff, I could activate and try to trigger the bug, that may could help you?

Thanks in advance

:-)

[Psy-Virus]

What I have observed in the last moths is, that it mostly happens when there is no junction pipe within a chunk. For example:

+ = Chunk with Junction pipe
- = Chunk without junction pipe

If you have something like that: +-+ The items often get stuck between the two junction pipes and flowing from one to another.

But I've never seen it if you have a junction pipe in each chunk. I'll observe this further and check further if this only happens if there's a chunk without a junction pipe between two with one...

Tough thing

[Psy-Virus]

Never mind, it happened again...

[Psy-Virus]

Okay, maybe my last theory was not completely accurate. I think that items only get stuck on pipes that go over chunk borders. Because: I just had something happen where 2 glowstone dust got caught between two interface pipes. And each of these pipes were in another chunk; Pipe |Border| Pipe

[Psy-Virus]

Found out something else. If items are stuck between junction pipes, newly autocrafted items get send to the position where the items are stuck.

[Psy-Virus]

After several more of those:

I am 99% sure that it only happens on pipes that cross chunk borders.

[MrTJP]

Queueing this as low priority, since I have plans to completely replace the routed pipe system in v5.

[Psy-Virus]

All right, I'll definitely keep an eye on it then.

Thanks a lot for keeping up your work! It's, with Dynmap, still the number one Mod on my servers.

If you need help with bug testing, feel free to mention me somewhere.

:)

[murilorodelli]

Hi, @MrTJP. I'm Playing the FTB Ultimate Reloaded Modpack v.1.8.0 on a Public Server and I'm having exactly the same issue. The 1.12.2 version already have the new code? Do you have plans to fix it?

[Psy-Virus]

Hey @murilorodelli, check this and this out. We're still on v. 4.9.xx

[murilorodelli]

Hey @Psy-Virus, @MrTJP

I already know what's happening. I've tried many setups until narrow it down. It's a chunk loading problem.

The items get stuck and bounce back exactly on the chunk border which led me to think this is a chunk loading order problem

In my case my network comprises 2 adjacent chunks (A, B), and the items have to travel from chunk A to chunk B. It works fine until the chunks get unloaded (ex. when log off the server). When the game loads back, the chunk B is loaded before chunk A (or vice-versa I could not simulate whats order causes the bug.) and the network don't get rebuild, causing the items to get stuck on the chunk border.

I've made the following test: I've put a double chest in the chunk border as a buffer, one side in each chunk and imported items from one side and exported from another. And that worked. The problem is that the chests interrupts the network

[Psy-Virus]

[...] which led me to think this is a chunk loading order problem

So maybe we need some sort of delay before the network recreates itself. Like checking first if all chunks have been loaded and then recreate the junctions. I guess this could be done before the server allows players to join(?)

Or another option would be to save the network as a scematic and on load, recreate the network with dummy junctions and let it assume that the junctions are there.

[Psy-Virus]

Oh wow, I missed that post for about 4 months. I'm so sorry :(

[MrTJP]

The network pipe system is getting a major rework on 1.15+. Since it is going to be completely re-done, so it doesn't make sense to track this issue anymore