redruin1 / factorio-draftsman

A complete, well-tested, and up-to-date module to manipulate Factorio blueprint strings. Compatible with mods.
MIT License
92 stars 17 forks source link

draftsman.env.update fails, path mangled on windows #98

Closed LiquidInsight closed 11 months ago

LiquidInsight commented 11 months ago

EDIT: I found an easy workaround: unzip the mod files! Hope this helps anyone else with this problem.

First, this package seems really wonderful, I'm very keen to a script that generates blueprints for my game. I'm having trouble at step 0 however, getting the environment to work with my mod list. I'm using Windows, and it seems like draftsman is mangling paths to use both nix and windows style paths.

Describe the bug I am calling draftsman.env.update(verbose=True) in Python. I have one mod (Industrial Revolution, 2.3.2) in the "C:\Users\danie.conda\envs\PrettySolar\Lib\site-packages\draftsman\factorio-mods" folder. When draftsman searches for sub-code files (i.e. code/data/globals in this case) it looks for a file at 'C:\Users\danie.conda\envs\PrettySolar\lib\site-packages\draftsman\factorio-mods/IndustrialRevolution/code/data/code/data/globals.lua' (checked with custom loader) , which doesn't exist because both forward and backwards slashes are getting mangled together.

Current Behavior Call draftsman.env.update() within a script (or alternatively, call "dratsman-update --verbose" at the command line).

Expected behavior Draftsman should stick to a single forward/backward slash convention when searching for files.

Additional context Add any other context about the issue here, if seemingly relevant.

Please also include the following: Draftsman version: 1.0.6 Python version: 3.9.7 Factorio version: '1.1.87'

If your issue is related to mod-loading with draftsman-update, in addition to the above information, please also include:

  1. The exact draftsman-update command that fails/produces incorrect behavior import draftsman.env draftsman.env.update(verbose=True)

  2. The error produced by the command (if the command does fail)

Discovering mods...

(zip) IndustrialRevolution 2.3.2

Determining dependency tree...

IndustrialRevolution 2.3.2 archive? True dependencies: base >= 1.1.37 ? DeadlockLargerLamp >= 1.5.0 ? DeadlockBlackRubberBelts >= 1.0.0 ? IndustrialRevolutionExtraComponents
? IndustrialRevolutionGas
? Dectorio
? textplates
? rso-mod

Load order: ['base', 'IndustrialRevolution']

SETTINGS.LUA: mod: IndustrialRevolution SETTINGS-UPDATES.LUA: SETTINGS-FINAL-FIXES.LUA: DATA.LUA: mod: base mod: IndustrialRevolution

LuaError Traceback (most recent call last) ~\AppData\Local\Temp/ipykernel_17140/1255214403.py in 9 import draftsman.env 10 ---> 11 draftsman.env.update(verbose=True) 12 # draftsman.env.update(verbose=True,path = r'C:/Users/danie/AppData/Roaming/Factorio/mods')

~.conda\envs\PrettySolar\lib\site-packages\draftsman\env.py in update(verbose, path, show_logs, no_mods, report) 1828 print("\tmod:", mod_name) 1829 -> 1830 load_stage(lua, mods, mod, stage) 1831 1832 # Reset the included modules

~.conda\envs\PrettySolar\lib\site-packages\draftsman\env.py in load_stage(lua, mod_list, mod, stage) 277 lua.globals().lua_add_path(mod.location + "/?.lua") 278 --> 279 lua.execute(mod.data[stage]) 280 281

lupa\lua52.pyx in lupa.lua52.LuaRuntime.execute()

lupa\lua52.pyx in lupa.lua52.run_lua()

lupa\lua52.pyx in lupa.lua52.call_lua()

lupa\lua52.pyx in lupa.lua52.execute_lua_call()

lupa\lua52.pyx in lupa.lua52.raise_lua_error()

LuaError: [string ""]:260: module 'code.data.globals' not found:no file 'code/data/globals' found in 'IndustrialRevolution' archive no field package.preload['code.data.globals'] no file 'C:\Users\danie.conda\envs\PrettySolar\lib\site-packages\draftsman\factorio-mods/IndustrialRevolution/code/data/code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\lib\site-packages\draftsman\factorio-mods/IndustrialRevolution/code/data/globals.lua' (checked with custom loader) no file 'code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\lib\site-packages\draftsman\factorio-data\core/code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\lib\site-packages\draftsman\factorio-data\core\lualib\code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\lib\site-packages\draftsman\code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\lua\code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\lua\code/data/globals\init.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\code/data/globals\init.lua' (checked with custom loader) no file '.\code/data/globals.lua' (checked with custom loader) no file 'C:\Users\danie.conda\envs\PrettySolar\code\data\globals.dll' no file 'C:\Users\danie.conda\envs\PrettySolar\loadall.dll' no file '.\code\data\globals.dll' no file 'C:\Users\danie.conda\envs\PrettySolar\code.dll' no file 'C:\Users\danie.conda\envs\PrettySolar\loadall.dll' no file '.\code.dll' stack traceback: [string ""]:9: in main chunk [string ""]:260: in function 'require' [C]: in function 'old_require'



5. The mod list with their versions used with the command
Industrial revolution 2.3.2. Can be found [here](https://mods.factorio.com/mod/IndustrialRevolution/downloads). Mod is 120MB due to graphics, so I won't attach it to this github issue. 
redruin1 commented 11 months ago

Thanks for the report, I'll take a look at this in a bit. The inconsistent path slashing is a culprit of 3 different conditions:

  1. Python's os.path utils will default to the current OS's preferred path, forward-slash on unix and back-slash on windows
  2. zipfile (the module used for loading the mod archives) doesn't work with backslashes, so everything has to be coerced to forward slashes for interacting with zip archive internals
  3. Finally, the process for finding Lua files uses Lua's package.path variable, which uses system-specific slashes; and these paths are combined with the previously normalized forward-slash paths to comply with zipfile, which is almost certainly the cause of the mixed pathing.

Up to this point, this mixed pathing has worked alright in terms of file resolution (if you could verify with a different set of mods/modpack that would be great), so if I had to guess there's probably something specific with Industrial Revolution that causes the breakage, or when I redid the pathing code I introduced a regression. I'll install the mods and see if I can reproduce.

redruin1 commented 11 months ago

Fixed:

Load order:
['base', 'IndustrialRevolution3Assets1', 'IndustrialRevolution3Assets2', 'IndustrialRevolution3Assets3', 'IndustrialRevolution3Assets4', 'IndustrialRevolution3']

SETTINGS.LUA:
        mod: IndustrialRevolution3
SETTINGS-UPDATES.LUA:
        mod: IndustrialRevolution3
SETTINGS-FINAL-FIXES.LUA:
DATA.LUA:
        mod: base
        mod: IndustrialRevolution3
DATA-UPDATES.LUA:
        mod: base
        mod: IndustrialRevolution3
DATA-FINAL-FIXES.LUA:
        mod: IndustrialRevolution3

Extracted mods...
Extracted entities...
Extracted instruments...
Extracted items...
Extracted modules...
Extracted recipes...
Extracted signals...
Extracted tiles...

Update finished.
hella slick; nothing broke!

(factorio-draftsman) D:\SourceCode\repos\Python\factorio-draftsman>python
Python 3.10.0 (tags/v3.10.0:b494f59, Oct  4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from draftsman.data import entities  
>>> entities.containers
['wood-pallet', 'tin-pallet', 'wooden-chest', 'tin-chest', 'bronze-chest', 'iron-chest', 'steel-chest', 'big-ship-wreck-1', 'big-ship-wreck-2', 'big-ship-wreck-3', 'blue-chest', 'red-chest', 'factorio-logo-11tiles', 'factorio-logo-16tiles', 'factorio-logo-22tiles', 'starter-crate-medium-1', 'starter-crate-medium-2', 'starter-crate-medium-3', 'starter-crate-medium-4', 'starter-crate-small-1', 'starter-crate-small-2', 'starter-crate-small-3', 'starter-crate-small-4']

Problem was that the mod location (which ever since folder loading got added is now important for archive resolution) wasn't being converted to forward slashes, meaning it would never be replaced with the internal archive folder. This is why archives in an external directory would work, and why extracting the files out of the archive also solved the problem. This will be fixed in 1.0.7 or 2.0, whichever comes first.