Not going down the rabbit hole of perfectly recreating the Factorio environment

KirkMcDonald/FactorioLoaderLib#3 addresses two more differences between how mods are processed as opposed to how Factorio does it. I think these and the ones already fixed are only the first of a whole lot of probably more subtle and harder to track down discrepancies. The examples I can think of right now:

Actual load order isn't covered by table.sort.
- Factorio uses Natural sort order instead of ASCII value based one (source)
- There seems to be special cases for core and base always loading first
- If a mod has multiple dependencies, in which order will these dependencies be called? Probably also in natural sort order, but that would need testing, since I couldn't find information on that.
Factorio's version is slightly different from normal Lua in order to support determinism in multiplayer. Most prominently modders can expect pairs() to be deterministic (by I think insertion order) while it won't be in the loader. These will probably not manifest themselves in hard crashes, but instead cause slightly wrong recipe lists (aka harder to find and fix).
Aside from the differences betweeen normal and Factorio Lua, there might also be problems with using Lua 5.3. A bit of a constructed example would be with the new integers: tostring(10.0) == "10" is true in 5.2 but false in 5.3.

Because of that I would suggest to use the Factorio environment itself rather than emulating it. Two options I see here:

Dynamically generate a mod that depends on all mods in the mod-list.json and executes it's code in data-final-fixes.lua. I don't know how well this would work when the game is running simultaneously (probably a desired feature). I think it is possible at least in the stand-alone version, I don't have any idea whether this would work with the steam build though.
Kindly ask Rseding if he could update the Lua version used by Factorio and use that instead of default Lua. IMHO less convenient to implement, but fixing the differences regarding mod load order is probably manageable.

It wouldn't be hard to implement a natural sort. I've avoided the problem so far because there are multiple ways of defining a "natural sort," and I'm not sure exactly what Factorio does; and because it has not seemed to matter so far, with the data I've thrown at the data-dumper. This is something I've thrown into the category of "solve it when it's a problem."

I think the only special case is the "core" mod being loaded first. The base mod comes next by dint of everything else depending (possibly implicitly?) on it. The loader manages this by giving base a dependency on core, and the dependency-ordering function does the rest.

That brings up another potential question, which is that the Factorio docs don't specify exactly how it topologically orders the mods. A given graph may have multiple valid topological orderings, even given the secondary natural sort. FactorioLoaderLib uses a topological ordering. Whether it's the same topological ordering that the game itself uses is not something I know, but it's been good enough so far.

In practice, I would not expect mods to depend on initialization order outside of the order implied by their dependencies. To the extent that this is true, it gives these tools a lot of wiggle room with respect to exactly duplicating the game's behavior.

Factorio's alterations to its Lua environment are more of an issue, but again, I'd put this in the category of "solve it when it's a problem." I'm not all that worried about it, and these things can be addressed as they're noticed.

Lua 5.2 vs. 5.3 is hairier. While Factorio nominally uses 5.2, it may have some bits from 5.3 folded in, in addition to its custom changes. In practice, just using stock 5.3 has worked for me so far. If the version incompatibility turns out to be an issue, the "correct" solution may be to backport these tools to Lua 5.2. I took a look at this earlier, and I think the main issue was settingloader.lua, which makes use of the string.unpack() function that exists in 5.3 and not 5.2. Replacing this and fixing any other issues that crop up, and getting the golua bindings switched over to the 5.2 branch, would not be a huge problem.

I would like to avoid using Factorio itself, if possible. For one thing, I really like being able to iterate on these tools more quickly than Factorio takes to load. I don't think the environment needs to be perfectly recreated. It only needs to be good enough, and problems, once found, can be solved. This will be more work than relying on the game itself, but I think it is much more convenient.

I didn't mean to say that that implementing natural sort specifically would be hard. There are even Lua implementations available here and here. In my post above I only thought of relatively simple graphs, but more complex graphs will indeed pose even more questions.

The core issue for the use case of the calculator really is the fact that things are underspecified. In general this is a good thing, mod authors should not concern themselves with what effectively are implementation details. Given the dependencies, Factorio will just make it work (or throw an error if it isn't acyclic I guess).

Adhering to dependencies and using alphabetical sorting will probably cover 98+% of cases. But I'm pretty sure eventually some combination of mods will come up where things are different. Not necessarily because a mod author is trying to be sneaky, but because of situations where multiple mods are combined in a way that wasn't anticipated by the authors. One of them might be doing changes in a generic fashion (e.g. iterate over all recipes with ingredient x and replace with y) that won't be applied if it's loaded in the wrong order. This is the kind of change I'm thinking of when I said hard to track down. Unless you go through every recipe by hand and compare it with the one in-game you won't find these differences until you've half build a factory and realize that the numbers don't add up.

For the major mod packs that are intended to be run together this probably won't be a problem because they most likely will have the dependencies set correctly. So anything you might eventually host on the official version of the calculator should be fine. And most people won't run a custom version on localhost anyway, no matter how easy it becomes.

If string.unpack() is all that is 5.3 specific, then indeed switching might be easy if needed. There are library based implementations for 5.2/5.1 that seem to only slightly differ in the supported flags in the format string, see here and here.

Good point on the load times, I did not think of that. The sprite atlas cache can be used to speed things up, but it will still be significantly slower than running a custom environment that only loads what is actually needed.

Since there isn't a concrete action coming out of this, feel free to close the issue whenever you think it's appropriate.

@KirkMcDonald Let me come back to this with some real examples of things going wrong now. Interestingly enough those are basically two of the three points I brought up in my initial post.

Non-deterministic `pairs()`

Industrial Revolution has some fairly complex code to generate several standard items for multiple ores. It's basically a meta-prototype system that generates base-game prototypes:

-- code/data/components.lua
return {
    ["ore"] = {
        type = "item",
        order = "aa",
        materials = {"wood","rubber","stone","carbon","copper","tin","iron","gold","uranium"},
        -- [...]
    },
    ["gravel"] = {
        type = "item",
        order = "b",
        materials = {"stone","carbon","copper","tin","iron","gold","uranium"},
        made_from = {
            ore = { amount = 1, result = 1 },
        },
        -- [...]
    },
    -- [...]
}

-- code/items-recipes/items-generated.lua
for component,componentdata in pairs(DIR.components) do
    for _,material in pairs(componentdata.materials) do
        -- generate the item definition for this material
        -- [...]
        -- generate the recipe definition for this material
        -- [...]
    end
end

Now the code that generates these prototypes assumes that the components table is processed in order, e.g. generation of gravel will depend on the prototype definitions for ore being created already, which is generally not true for LoaderLib. This now manifests in attempt to index a nil value crashes. The solution a the moment is just to try often enough until you hit the one case where randomly everything gets processed in the order it was defined in.

The same problem used to exist in Alien Biomes (required dependency for e.g. Space Exploration), but something must have changed because it is no longer the case and I don't know which version I saw it with.

Integer types in Lua 5.3

print(string.format("%d", 13.37))

In Lua 5.2 and Factorio, this will print 13, in Lua 5.3 this is an error: bad argument #2 to 'format' (number has no integer representation). This is also actively happening in Industrial Revolution (see code/items-recipes/recipes-scrapping.lua:60) and appears as soon as you get past the first issue.

I've played around with Factorio's command line arguments and I think it is worth giving the idea of running the game itself a second try to reduce maintenance costs. If I read your comment correctly your main concern is the increased time it would take to scrap data.raw.

Most of startup time can be removed by telling the game to only start what the calculator actually needs. The most important things here are the following two:

Start the game in headless mode and never load the sprites. This can be achieved via e.g. --start-server-load-scenario base/freeplay. I have verified this to work with both Steam installs as well as archives from the website.
Disable prototype history via --disable-prototype-history. This is used in tooltips to display which mods have modified a certain prototype. The history is generated by diffing the entirety of data.raw after each loaded mod for each of the three stages. Removing this cuts down data-stage execution time for Seablock by more than 90% for me.

With these two and some smaller things I can get from starting the executable to a game being hosted in less than one second for vanilla and less than two seconds for Seablock.

Based on this I've put together a minimal POC implementation of a data scrapper mod that just dumps the data.raw JSON in data-final-fixes.lua. Very crude performance numbers below:

	`./PrintData.lua`	Factorio
Seablock	≈ 5.1s	≈ 7.0s
Vanilla	≈ 3.7s	≈ 4.4s

There is still a clear performance hit, but I think those times would be acceptable for the advantage of never having to deal with all the potential problems of executing mod-code directly. What do you think?

In all cases most of the time is actually spent serializing to JSON. Maybe there are some possible optimizations, but that would be a different point entirely.

KirkMcDonald / factorio-tools