Interface for Machine Learning

[MortenTobiasNielsen]

@theis999 I was thinking about investigating what would be needed for people to be able to do reinforcement learning (or ML in general) on Factorio. I think it would be cool to see what people would be able to do. it seems like factorio can be run in some sort of headless mode Factorio Headless mode, which most likely would make it possible to run multiple games in parallel, which would be needed for fast generations.

What do you think? :)

[theis999]

I am not very interested in ML myself but i could see others might be. It might be worth it to ask if anyone would be interested first. In addition to the headless mode, it is possible to: --load-game FILE --until-tick TICK

So it could be possible to run smaller segments, we would need some sort of output to perform optimizations on which can be done using script output.

I believe it is possible but it will not be my highest priority.

[MortenTobiasNielsen]

Do you have an idea of a forum where the type of Factorio players which might be interested in something like that would be? :) Nice - yeah I guess those commands would be very useful.

I am not familiar with script output where can I read about it? :)

Fair enough - I will see if I find the motivation and time to pursue it.

[theis999]

reddit_technical reddit forum and the steelaxe and technical-factorio discord seems like the most obvious places.

Script output is a folder where mods can write to, usually placed here ~\AppData\Roaming\Factorio\script-output. I used it to extract files for the entity bound_boxes. It should be possible to bind log/debug/error output to it too.

[theis999]

Finding a good fitness function is very hard for factorio, as very few actual values in the game gives an accurate depiction of good progress. Which makes me think it will be hard to near impossible, to get ML to generate a full run.

However i can see a use case a partial solution, Walking. Finding a good walking path is a hard problem, which consumes a lot of user time.

[MortenTobiasNielsen]

I would guess some aggregate of production counters and research would need to be used.

Agreed - finding the best walking path between multiple steps can be challenging, it would be nice to have a "I need to start here (x, y), do these steps and stop here (x, y). This is though a famous problem (the travelling salesman), which can be nearly impossible to solve within a reasonable amount of time, when the complexity increases, which often would happen in Factorio.

[theis999]

True, and minimizing the amount of stuff in inventory. As you want stuff placed and generate value.

Good point, still. The biggest problem is getting a path through buildings. As long as the pathing is short enough then even a bad algorithm will do😎

[MortenTobiasNielsen]

Hehe, so we need to optimize two very common operations research problems - production optimization and network optimization, but together. 😅

True, even a bad algorithm can do simple stuff, but so can most players. I don't guess it is optimizing the next 5-10 steps, but mostly the next few minutes, which is the difficult part. Just setting up the parameters for such an algorithm would in most cases be very challenging - It is pretty much at college degree, well to truly understand those types of problems and optimize them efficiently. 😂