Factorio Annealer

The Factorio annealer is a program that generates a cost optimized layout of blocks for a factory given a set of items to be produced. Cost optimization is performed using a common place-and-route technique called simulated annealing. Simply put, a random move is generated and is conditionally accepted based on a given cost function. After a large number of moves, the design is considered optimized. A blueprint is then generated by this layout that can be directly ported into Factorio. Sets of sub items can be partitioned and all ingredients will be dedicated to make those items. An example 250 science/min factory from this program is provided. The example layout is shown below and the blueprint string is located in ./data/example_blueprint_250spm.txt. The first ~90 seconds of the example running is shown below. In addition, the first 90 minutes or so sped up by ~60x can be seen here.

Background

This idea has been floating around in my head for some years now, but I hadn't found a good use case for it until recently. The main issue with this idea initially was using large city blocks basically trivialized product generation. Meaning a single block could potentially generate all the needed product for a very large (i.e. 1k science/min) base. This made the problem of annealing a small number of large blocks not very interesting and could probably be done manually to a high degree of accuracy. It wasn't until the spring of 2022, I saw a post on the Factorio subreddit about a city block implementation that was smaller than the typical chunk aligned one. This got me thinking about a few questions. How small could I make these blocks? Would simulating annealing produce interesting results? These questions was finally what got me to start this journey of creating 57 block blueprints and writing this annealer.

Additionally, shortly after starting writing the program, redruin1 posted an Alt-F4 blog post about his project to programmatically generate blueprints called Factorio Draftsman. This was very fortuitous because it would save me the trouble of writing a program that imported a blueprint book, stitched hundreds (>200k total entities) of them together and then converted it back to a blueprint string to be imported into Factorio.

Top Level Organization

At the top level, this program is comprised of 5 parts: factory, annealer, blueprinter, micro-block blueprints, and factory drawer. The Factory class contains all item and recipe requirements and keeps track of the placement of the individual blocks. The Annealer class takes the factory layout and performs the simulated annealing algorithm, including move generation, move evaluation and cost tracking. The Blueprinter utilizes Factorio Draftsman and directs the creation of the blueprints of the factory. The micro city blocks is a collection of blueprints that implement the required recipes for science pack production and research. Finally, the FactoryDrawer class is a debug tool to visualize the current layout of the factory.

Factory

The Factory class stores all related data concerning item requirements and physical placement. The factory placement is done using FactoryBlock objects which describes the physical implementation of a recipe. These contain one or more FactoryCell objects which translate to a single physical cell of the factory. The FactoryCell contains FactoryCellIO objects that specifies an item or fluid, whether it is an input or an output and which train station it is assigned to. The factory also has pins which are raw resource interfaces with an external train/belt system. Each solid item pin assumes an input of 2 saturated blue belts (5400 items/min) and each fluid pin assumes 25000/min.

The factory generates an item list and recipe list from vanilla game data by parsing the corresponding Lua files from the official Factorio data repository. This is not a robust method, but learning to load all game data into Lua first and then extracting recipes and items from there was not something I wanted to do. If this were to be implemented, this program could be extended to include mods and their items and recipes. An important point to note here is that for items that have multiple recipes, each item can specify which is the preferred recipe. In vanilla Factorio, this isn't a huge deal, but mods tend to have many different recipes that can generate the same item (i.e. wood, oxygen, rocket fuel in Krastorio 2). In addition to vanilla items and recipes, some custom items and recipes are added to allow for ease of use in dealing with edge cases (top and bottom of recipe tree). For example, a special labs recipe is created to handle the research and takes in all of the science packs as inputs with no outputs.

Each relevant recipe was implemented in game and stored as a FactoryBlockTemplate. The templates specify required ingredients, products and at what rate along with some physical information regarding the placement of the item requests/production. Some templates skip certain intermediate ingredients to avoid transporting too high of a volume of product. The most notable example is copper wire. For electronic circuit production, far too much copper wire would need to be delivered and many blocks struggle with offloading the ingredients (i.e. armor-piercing ammo, electronic circuit, etc.).

Item requirement structures are implemented using a Partition class. Partitions are a way to decouple cell dependencies in one tree from another. Using default factory options, the only partition implemented is the labs recipe. With a single partition, nothing special is implemented. When additional partitions are added, the item requirement calculations are performed on a per partition basis. The program can also be configured to implement separate LTN networks per partition. This will restrict which cells that can supply another. For example, if we have a factory that has a electronic circuits and armor piercing ammo partitions, the amount of iron and copper plates are calculated separately. The iron and copper plate blocks for electronic circuits will only supply electronic circuits and not armor piercing ammo and vice versa.

The program implements a simple, non-matrix based, calculator to generate the item requirements given the desired item production. This works well for all items that only have a single recipe used. In vanilla Factorio, this breaks down for oil products. Petroleum gas can be generated via oil processing and light oil cracking. How much to allocate to each of these recipes must be handled differently. To handle these recipes, the calculator takes a bottom down approach. First, the amount of refineries for heavy oil is calculated. Second, light oil is calculated by first subtracting out excess light oil from heavy oil generation and then calculating the number of refineries are needed to cover the remaining light oil with all extra heavy oil cracked into light oil. Finally, petroluem gas production is calculated similarly to light oil. The astute observer can tell this is not a very robust method to solve this problem. If ever the amount of heavy oil or light oil is high enough that the excess petroleum gas is greater than the petroluem gas requirement, an excess will accumulate. This would result in an eventual dead lock caused by petroleum gas completely filling up. A matrix based solver would be able prevent this kind of thing by diverting some petroleum gas into solid fuel, but alas, I'm too lazy to implement it.

Micro City Blocks

The factory blueprint is enabled by a library of custom cells I've coined as Micro City Blocks (MCB). These are minimum sized train-based blocks that will allow 2 1-1 train stations (1 on top, 1 on bottom) that can directly feed into each other. All in all, each cell is 36x32 tiles in size. Each FactoryBlockTemplate has an associated blueprint within this library. For example, the block for electronic circuits is shown below. The train track grid uses right hand drive signalling rules and due to the small size of each block and the desire to pack these as close together as possible, intersections were implemented without left turns (see below). Therefore, in order to turn left, a train must go straight and make 3 rights. While this may be a bit longer distance wise, having left turns within the intersection would make it so any train entering the intersection block all other trains trying to enter the same intersection. Meaning, a train going straight would block a train going the opposite direction. This issue outweighed the need for left turns. The train scheduling is done using the Logistic Train Network (LTN) mod to minimize the number of trains running at any given moment. There are a total of 57 defined blocks, one for each recipe within the research recipe tree and a few auxiliary blueprints for LTN and resource interfaces at the edge of the factory. This set can be extended to implement any item's recipe tree, but the current blueprint book contains on those related to the vanilla science packs.

Annealer

The simulated annealing algorithm is implemented within the Annealer class. The main algorithm loop is quite simple: generate, evaluate and implement a move. A move is simply selecting 2 FactoryBlocks to swap. Move generation is random but is constrained by blocks that contain more than 1 cell and by the number of adjacent LTN depots. Factory blocks with more than 1 cell require the move generation to check for boundary cases in the other set of factory blocks being swapped (at edge). Within the factory options, the user can specify the minimum number of LTN depots each cell must be adjacent to. This means that for a value of 2, every cell must have 2 LTN depots adjacent. Valid options are 0-6. Edges require half the adjacent LTN depots and corners require none.

Once a valid move is generated, the algorithm evalutes the change in the cost function of the factory. If the change results in a lower cost, the move is accepted. If it incurs a higher cost, it is randomly accepted based on the change difference and temperature of the algorithm. Simulated annealing starts with a high temperature. This means that for early moves, a lot of higher cost moves will be accepted. As the algorithm progresses, temperature falls and the probability that a bad move is accepted exponentially declinces.

The cost function consists of a simple distance calculation between origin and destination that the train would have to traverse and a function of the volume of trains that is expected based on the producer to requester relationship (i.e. electronic circuits to processing units is high, iron gear wheels to satelite (radar) is low). This volume is then divided down by the amount of stations that are identical to the one in question. For example, electronic circuits to processing units has a high volume, but if there are 100 processing unit blocks, the volume of trains to any individual block is divided by 100. If a move is accepted, the factory implements this by swapping the sets of blocks. This loop will continue until either a specified number of iterations has occured or the average change of the past 1000 moves is below the given function tolerance.

Blueprinter

As mentioned in the introduction, the ability to export the design into a valid Factorio blueprint was made possible using Factorio Draftsman and the imported MCB blueprint book. Factorio Draftsman is an extremely powerful tool. It allows the user to create a blueprint and add entities or modify existing entities within a programming environment. If you visit the Alt-F4 page about it, you'll see a number of examples of what can be done with this tool including creating map images, ROM for a computer, preloaded turrets, etc. If you've ever had a Factorio related idea that needs a blueprint created, Factorio Draftsman will fulfill all your needs.

After my shameless plug for Factorio Draftsman, back to the program at hand. After importing the MCB blueprint book, each one is placed inside of a custom factorio-draftsman Group object. Doing so allows the entities within a single blueprint to be handled all at once with minimal overhead. When adding a Group to a Blueprint, draftsman makes a deepcopy of the Group, including each entity and any circuit connections and filters. When adding the same block multiple times, this lets you simply set the position of one block, add it to the blueprint and then set the position for the next block without having worry about the first block's position being modified.

Blueprinter generates a single blueprint of all cells and the connecting train track grid and saves the blueprint string to ./data/factory_blueprint.txt. In addition, it should create all needed power connections between each of the cells. It doesn't always work, so make sure to double check it. For example, the example factory provided has about 98% connect correctly, there poles along the bottom left not connect. For larger factories, it is probably quicker to not call the remove and add power connections functions, but instead place the blueprint, delete all big electric poles and then replace them all using the main train track grid blueprint. A bit tedious, but it will definitely be a faster turn around, since the 250spm example provided too about 30 minutes to generate the power connections.

Factory Drawer

The FactoryDrawer allows the user to visualize the factory layout. It generates a grid and places the item icon into the space it is placed. Below is an example of this in action. This is a powerful tool in debugging factory cell placement, cell dependencies and annealing. It also allows the user to visualize a proposed move generated by the annealing process and contributing cells that affect the cost of the move.

Initial Layout Before Annealing

Optimized Factory Layout

Program Options

The program provides options for the Factory and Annealer classes which will modify the behavior of the program and precision of the result.

Factory Options

Here are the available options that can be modified. See this for examples of the effect of modifying many of these options.

• "top-items" : Top level item names and production rates for the factory to produce. These cannot be a dependency of each other or of any of the top level items of the partitions. Assignment and addition will check ensure this criteria is met.
• "depot-adjacency-requirement" : Factory layout requirement for the number of LTN depots adjacent to each FactoryCell. Default is 2, maximum is 6.
• "productivity-bonus" : Productivity bonus from a single module. Vanilla bonuses are in [0, 0.04, 0.06, 0.1], default is prod-3 (0.1)
• "calc-exceptions" : List of solids/fluids to be handled differently when calculating the factory requirements. These will generally be items that can be produced via multiple recipes. For vanilla Factorio, this includes oil products, i.e. light oil and petroleum. Default items are heavy oil, light oil and petroluem gas. If this is modified, the user must modify the factorycalculator.calculateNormalizedRequirements function to handle these items. Additionally, the prerequisites to these items are not calculated. Currently no functionality supports dealing with prerequisites of calculation exceptions (vanilla only has crude oil which is a resource and has no prerequisite itself).
• "partitions" : List of all item names as top level partitions. A partition is defined as a set of factory blocks that produce the resources needed to create the top item. The size and requirements for these partitions are calculated based on the desired rate of the factory's top level items defined in "top-items".
• "partition-pins" : Specifies whether the pins will be global or partition specific. If "use-unique-network" is set, the pins will be limited to supplying the partition only. Otherwise, the pins will be laid out per partition instead of all together globally. Not recommended without unique networks.
  • "block-template-path" : Custom list of all block templates to be used in the factory. For any partition, all sub-recipes must be defined within the given file. If any block skips an intermediate product (i.e. electronic circuits request copper plates and makes copper wire onsite), that intermediate product MUST be skipped for ALL other recipes. The block templates do not specify if they skip products at the moment. During the calculation of factory requirements, the vanilla recipes are modified based on the provided block templates. Having a block for an intermediate product that is skipped will break this functionality.
  • "use-unique-network" : Boolean specifying if the LTN train stations should operate using unique network ids. This means blocks within a partition will only service blocks within the partition unless the item is the top item in a partition. For example, if advanced circuits have their own partition, those factory blocks will have to be assigned to the advanced circuit partition blocks to get the needed resources and also assigned to any other partition network that uses them, i.e. chemical science packs.
  • "aspect-ratio" : The x:y ratio of the factory. This will try to size the factory to have aspect-ratio times more/less columns than rows. A value of 2 will cause it to be short and wide, while a value of 1/2 will make it tall and skinnier.
  • "pin-padding" : The number of empty cells adjacent the corners for pin placement. The default of 1 means there will be one empty FactoryCell adjacent to the corners where no pin is placed. If value is too large, it may result in the inability to place all pins in allowed spaces.

Annealer Options

Here are the available annealer options:

• "initial_temperature" : Specifies the initial randomness of the annealing algorithm. The acceptance probability when a move increases the total cost function of the factory is calculates as exp(-cost_change/temperature). For large temperatures, the probability is near 1 and will almost always accept bad moves. The temperature decreases exponentially by the function initial_temperature/iteration. Therefore, as the algorithm progresses, the probability to accept bad moves decreases and will eventually only accept moves that reduce the cost function. Default is 1000
• "moves_per_iteration" : This specifies the number of moves to perform each iteration of the algorithm. Once the iteration is incremented, the current temperature is recalculated. Default is 50.
  • "max_iterations" : Gives the algorithm a hard stopping point of iterations if the function tolerance cannot be met. Default is 10000.
  • "function_tolerance" : Defines the accuracy of the annealing algorithm. If the average of the previous 1000 moves is less than the function tolerance, the algorithm will end. Higher values will result in a less optimal solution while smaller values will result in longer runtime and should produce a more optimal solution. For smaller factories and high partition count (partitioned factories are basically small factories within the larger whole), it is recommended to reduce this value. The cost of a move is not normalized in any way, so the cost change in a large factory will always be larger than that of a smaller one. Default is 0.5.

Installation

I'm not very experienced with regards to setting up code that can be taken and ran by others, so this installation guide will not be optimal in the slightest.

1. Download this repository
2. Install prerequisite data and programs.
   1. In the Factorio Annealer folder, clone into the official Factorio data repository
   2. In a different location, clone into the Factorio Draftsman repository (current working version is 1.0.2)
   3. Copy the ud.sh and up_script.py from the data folder into the Factorio Draftsman main folder
   4. Edit the ud.sh and up_script.py as specified inside each.
   5. Download the following mods
      1. Logistics Train Network
      2. Inventory Sensor
   6. Run the ud.sh shell script. This will install Factorio Draftsman to the site-packages folder as an egg file. It will also copy over mods from the specified location into the installation directory and run the Draftsman update script. It is important to copy over mod-list.json and mod-settings.dat as well as the zipped mods as they are in the Factorio directory.
   7. Install the following Python modules
      1. lupa
      2. matplotlib
      3. pillow
      4. numpy
3. Add the path to the egg file into the environment variables
4. Open run_game.py and modify the factory and annealer options

5. Run python run_game.py

   That should do it, but I'm sure it won't work like it did for me, so just open an issue and I'll help debug the issue.

Misc. Information

This section is for other bits of information that are important for the user, but didn't fit anywhere else.

• The train fuelling is handled by bots. Each LTN depot has a requester chest that request 10 rocket fuel. In order to start the factory, it will need to be externally supplied with fuel via the logistics network.
• It is recommended to download the Train Mode Switcher mod. The blueprints generated by this program places a number of 1-1 trains for you, 1 or 2 for each LTN depot cell that is placed. This can result in a few hundred trains and this mod will let you select every one and switch them all to automatic mode.
• The Inventory Sensor mod by Optera (LTN author) is included to help prevent issues when unloading fluids from trains. If some fluid remains in a train's wagon, it will be unalbe to fulfill subsequent requests. The inventory sensor gives a more accurate value to the pumps and keeps them active for longer before the train is disconnected.
• The initial factory layout alternates between rows of various factory cells and rows of LTN depots. This method produces an excess of depots, but the annealing process will push those away from the high value locations. An excess also allows for the user to try different numbers of required adjacent depots. For this application, I believe only 1-2 is required to service the factory.
• The example factory used the following non-default settings:
  • top-items: 250 labs
  • partitions: All science packs
  • paritioned-pins: True
  • pin-padding: 3
  • function-tolerance: 0.2