Introducing "Pattern Provider P2P Tunnel" for Efficient Pattern Distribution

[TinouHD]

Describe the feature

The "Satellite Pattern Provider" is an enhancement to the current Pattern Provider functionality in Applied Energistics 2. The concept is to create a master-slave relationship between a primary Pattern Provider and multiple Satellite Pattern Providers. This feature will allow patterns stored in a master Pattern Provider to be automatically propagated to all connected Satellite Pattern Providers, simplifying the process of pattern management and distribution across large ME networks.

How It Should Work:

1. Master-Satellite Linking:

   • The linking process involves maybe a new card to add the satellite linking GUI to a existing Pattern provider. The linking process work the same as the wireless linking scenario.

2. Pattern Synchronization:

   • Patterns inserted into the master Pattern Provider will be automatically copied to all linked Satellite Pattern Providers.
   • Any updates (addition, removal, or modification) to the patterns in the master Pattern Provider will be instantly reflected in all linked satellites.

3. Crafting Jobs Distribution:

   • When a crafting job is initiated, the system will consider both the master and satellite Pattern Providers, allowing for better distribution of crafting tasks across the network.
   • Satellite Pattern Providers will function as extensions of the master, providing the same set of patterns to connected machines.

4. User Interface:

   • Satellite Pattern Providers will have a simplified interface indicating their linked status and the master they are connected to.

5. Error Handling:

   • If a Satellite Pattern Provider loses connection with the master (e.g., due to a broken cable), it will display an error status in its GUI.

Benefits:

• Centralized Management: Simplifies the process of managing patterns in large and complex ME networks.
• Efficiency: Reduces redundancy and ensures consistency of patterns across multiple machines.
• Scalability: Facilitates the expansion of automated crafting setups by allowing easy addition of new machines and Satellite Pattern Providers.

Use Case Scenario:

1. Imagine a scenario where you have a large factory setup with multiple furnaces, pulverizers, and other machines. Instead of individually inserting the same pattern into each Pattern Provider connected to these machines, you can use a master Pattern Provider linked to several Satellite Pattern Providers. By adding or updating a pattern in the master, all connected machines through the satellites will automatically receive the new pattern, ensuring seamless and synchronized operations across your entire network.
2. Imagine a world with GregTech 👀

This enhancement will significantly streamline pattern management in AE2, making it more user-friendly and efficient for large-scale automation projects.

Reasons why it should be considered

1. Enhanced Usability: This feature greatly simplifies the user experience by centralizing pattern management, reducing the need for repetitive manual updates across multiple Pattern Providers.
2. Design Consistency: The Satellite Pattern Provider aligns well with the core design principles of AE2, emphasizing automation, efficiency, and scalability within the ME network.
3. Community Demand: Many users have expressed the need for more efficient ways to manage patterns, especially in large-scale setups. This feature addresses a common pain point, potentially increasing user satisfaction and adoption of the mod.
4. Future-Proofing: As players create increasingly complex networks, the need for scalable and manageable solutions becomes critical. The Satellite Pattern Provider future-proofs AE2 against these evolving requirements, ensuring it remains a robust solution for automated crafting and processing.

Additional details

No response

[Enzoxdt]

This alredy ''exists'' if you press G on the pattern provider there is an explanation how 1 provider can feed multiple machines ( up to 7), the problem is that the round robin sucks, i don't know if it is a me problem not knowing how to configurate the pattern provider right or if its just sucks

[TinouHD]

Yeah, I know that you can use this trick with an interface behind a pattern provider but my idea is more acting like if you have multiple provider with the same pattern on different machines. (e.g. I use AE2 with my gregtech setup in ATM9 and for speeding up the process I have 3 Pattern providers with exactly the same patterns connected to 12 machines. And it's a lot of work to update all 3 pattern providers when I need to add or change a recipe)

[Geccpls]

This feels like something you can solve with subnets; Here's an example of my current setup. This lets me only make a single recipe and put it in a single pattern provider, and have the recipes handled by multiple machines:

But that's not very clear so I recreated it in a way that's easier to look at:

Purple boxes indicate Storage Busses. Light blue boxes indicate Interfaces. The machines in this example are Industrial Revolution Electric Furnaces, pretty standard stuff. Each colour cable indicates a different network / subnetwork. (Note that there are green and lime cables in this image; The bottom two on the left are lime.)

The blue network can request up to 18 types of items be smelted. When a request is processed, the pattern provider bus will attempt to store the items in the green subnet; The green subnet then stores it in the furnaces, which are configured to use their left side as an input side, and then they start processing. They're also set to auto export to the right side, so they will insert into the interfaces on the red subnet, which storage-busses them back into the main network.

I've also included an example of how you can recurse subnets; The green subnet stores items in the lime subnet, which can in turn store items in further furnaces. Using additional subnets in this way lets you distribute to an arbitrary number of machines without making a new controller network.

All in all, it's very possible to make a single pattern provider push to multiple machines- You can even make multiple pattern providers push into multiple, shared machines like in the example. Different crafting units can request different recipes from these pattern providers simultaneously. (e.x. Suppose a single pattern provider with recipes for smelting iron and copper ore. You could submit two crafting requests, for 32 copper and one for 128 iron. One furnace would receive 32 copper, and the other two furnaces would receive 64 iron.)

[TinouHD]

Thank you for answering. Yeah, I already use similar setups for some processing (mainly for chargers) but this setup doesn't work for all processing recipes. In Gregtech there are a lot of recipes that need many ingredients (like 5 or 8 items/fluids) with a certain amount for each ingredients. To avoid some problems it is recommended to insert only 1 recipe at a time because many recipes share some ingredients (in different quantities). Also, with the setup you showed, crafts aren't round-robin across all available machines, the input of the first machine needs to be saturated before applied start to use the second (for similar items), this is not acceptable in Gregtech 'cause some recipe takes up to many long minutes !

For the moment, the solution to my problem is to have 1 interface per machine and create 1 pattern for each craft for every interface I have. And this is very annoying when you have 16 machines with the same crafts and you want to add a new craft even if you use a pattern access terminal...

[Geccpls]

Ah, yeah I see what you mean. I haven't figured out a generic way to handle that yet. I've been playing with Modern Industrialization a lot recently, and its machines lets you filter and set the max stack size for its inputs, which does help with the partial ingredients problem, but having a more generic solution would be a lot nicer.

One thing I tried and was hopeful about but didn't have success with was setting the pattern provider blocking mode to "do not push crafting ingredients if inventory contains a pattern input" while pushing into a subnet via an interface, which stored items in machine inputs; I was hoping if I set the storage busses to insert only it might go one machine at a time, but instead it just kept inserting into the first machine. Not really a surprise, but disappointing.

I think I would prefer a solution that expands on subnet functionality in such a way as to make this solvable rather than a block with the direct intent of solving it- Something like an option on pattern providers to push into separate inventories or all the same one, or a storage bus that only allows storing up to a certain amount. Or idk, a stocking bus that does can do priorities.

That being said, something like a satellite pattern provider is created, I'd like to propose a slight change: Instead of introducing a whole new block, add a Pattern Provider P2P Tunnel! The primary pattern provider acts as the 'input' side of the tunnel.

Another near-solution would be to rename the pattern providers in an anvil to guarantee they all show up next to each other in the pattern access terminal so you can dump recipes into each one.

[TinouHD]

Changing storage buses behavior isn't a solution, they are very effective at their job.. storing things outside of the network. Round-robin storage bus would make them useless to store things. And make them configurable is more confusing on their real utility.

The idea of a Pattern Provider P2P Tunnel is much more is the mood of AE2 and I like this idea ! Must determine how they would act. My idea is when putting a P2P on a pattern provider (if the P2P is the master one) it artificially extends the pattern provider acting like if the pattern provider has more faces for every connected "slave" P2P.

Yeah renaming provider is useful, but when you have let's say 16 extended pattern provider (from Extended AE) with the same name with 4 rows of patterns each, this is not a practical near-solution :x

[Geccpls]

My idea around interface/subnet changes was mostly based on how a lot of AE machines interact with interfaces already; They don't input into the interface, they skip the interface inventory and push directly into the machines. But I think I agree that changing this behaviour probably isn't viable.

I like that idea for how the Pattern Provider P2P works.

I kind of wonder if at this point it might be better for you to just use dedicated machines for each recipe, and just disable them with level emitters as necessary, rather than try to use request crafting? That is the approach i'm working on right now, haha.

[TinouHD]

I quickly looked at the code of implemented P2P tunnels. I'm not really sure how they work, but with further investigation, I might be able to implement this idea of the "Pattern Provider P2P Tunnel" that I described above myself. I've read the guidelines for contributing, and I'm not sure if this feature would be considered a major one. For now, I'm keeping the idea of implementing this myself in mind. I would really appreciate it if more people could comment on the idea.

Here is the updated feature request. Not sure if posting this here is the best or if I should open a new issue and close this one ?

Describe the feature

The "Pattern Provider P2P Tunnel" is an enhancement to the existing Pattern Provider functionality in Applied Energistics 2. This feature introduces a peer-to-peer (P2P) tunnel system that extends a single Pattern Provider across multiple machines, facilitating more efficient and dynamic crafting processes. By placing one P2P Tunnel on the Pattern Provider and linking it to multiple targeted machines using Memory Cards, the system treats each connected P2P Tunnel as an extension of the original Pattern Provider, effectively distributing crafting tasks while respecting the "Blocking mode" of the Pattern Provider.

How It Should Work:

1. P2P Tunnel Configuration:

   • To set up the system, place one P2P Tunnel on the Pattern Provider and additional P2P Tunnels on each targeted machine.
   • Link the Pattern Provider P2P Tunnel to the targeted machines' P2P Tunnels using Memory Cards.

2. Adding Patterns

   • Simply insert patterns into the Pattern Provider.

3. Crafting Jobs Distribution:

   • When a crafting job is initiated, the Pattern Provider will distribute tasks across all connected machines using a round-robin approach.
   • The system ensures that each machine is treated as an extension of the Pattern Provider, maintaining balanced workload distribution.
   • The "Blocking mode" of the Pattern Provider is respected, preventing the insertion of items/fluids into a machine if the machine isn't empty.

Benefits:

• Centralized Management: Simplifies the management of patterns by allowing a single Pattern Provider to control more machines.
• Efficiency: Enhances crafting efficiency by dynamically distributing tasks, reducing bottlenecks and improving throughput
• Scalability: Facilitates the expansion of automated crafting setups by allowing easy addition of new machines to the network.

Use Case Scenario:

1. Imagine a scenario where you have a large factory setup with many Pattern Providers having exactly the same patterns on multiple furnaces, pulverizers, and other machines. Instead of individually inserting the same pattern into each Pattern Provider connected to these machines, you can use a single Pattern Provider linked to several machines using P2P Tunnels. By doing so, all connected machines can be used by the Pattern Provider, ensuring synchronized and efficient crafting operations.
2. Imagine a world with GregTech 👀, many crafts, all chained up and very long processing crafts... Scaling is the way to go to reduce waiting times!

This enhancement will significantly streamline pattern management in AE2, making it more user-friendly and efficient for large-scale automation projects.

Reasons why it should be considered

1. Enhanced Usability: This feature greatly simplifies the user experience by centralizing pattern management for many machines at once, reducing the need for repetitive manual updates across multiple Pattern Providers.
2. Design Consistency: The Pattern Provider P2P Tunnel aligns well with the core design principles of AE2 (much more than my first idea !), emphasizing automation, efficiency, and scalability within the ME network.
3. Community Demand: Many users have expressed the need for more efficient ways to manage patterns, especially in large-scale setups. This feature addresses a common pain point, potentially increasing user satisfaction and adoption of the mod.
4. Future-Proofing: As players create increasingly complex networks, the need for scalable and manageable solutions becomes critical. The Pattern Provider P2P Tunnel future-proofs AE2 against these evolving requirements, ensuring it remains a robust solution for automated crafting and processing.

Additional details

No response