search

SFTtech / openage

Free (as in freedom) open source clone of the Age of Empires II engine 🚀
http://openage.dev
Other
12.72k stars 1.12k forks source link

Asset converter - missing slp? #1391

Closed wlata closed 3 years ago

wlata commented 3 years ago

Hi everyone,

It's my first time at Github, so apologies in advance if I'm doing something in a wrong place etc etc.

Recently I built the game and tried to convert the assets from Age of Empires TC w/ FE installed, Age of Empires HD 2013 (both the newest version and 4.3 - as some do recommend) and AoE2 DE. All failed.

The output of "make run":

Please enter an AOE2 install path manually.
> /run/media/user/7A58CF8158CF3AA1/Steam/steamapps/common/Age2HD
converting from '/run/media/user/7A58CF8158CF3AA1/Steam/steamapps/common/Age2HD'
INFO [py] Compatible game edition detected:
INFO [py]  * Age of Empires 2: HD Edition
INFO [py] converting metadata
INFO [py] [0] palette
INFO [py] [1] empires.dat
INFO [py] Starting conversion...
INFO [py] Extracting Genie data...
INFO [py] Creating API-like objects...
INFO [py] Linking API-like objects...
INFO [py] Generating auxiliary objects...
INFO [py] Creating nyan objects...
INFO [py] Creating requests for media export...
INFO [py] Starting export...
INFO [py] Dumping info file...
INFO [py] Dumping data files...
INFO [py] Exporting media files...
Traceback (most recent call last):
  File "run.py", line 15, in init run
    main()
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/__main__.py", line 138, in main
    return args.entrypoint(args, cli.error)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/game/main.py", line 71, in main
    used_asset_path = convert_assets(
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/main.py", line 100, in convert_assets
    for current_item in convert(args):
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/tool/driver.py", line 30, in convert
    yield from convert_metadata(args)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/tool/driver.py", line 91, in convert_metadata
    ModpackExporter.export(modpack, args)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/export/modpack_exporter.py", line 52, in export
    MediaExporter.export(modpack.get_media_files(), sourcedir, modpack_dir, args)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/export/media_exporter.py", line 68, in export
    export_func(request, sourcedir, exportdir, **kwargs)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/export/media_exporter.py", line 287, in _export_terrain
    media_file = source_file.open("rb")
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/util/fslike/path.py", line 112, in open
    handle = self.fsobj.open_r(self.parts)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/util/fslike/wrapper.py", line 49, in open_r
    fileobj = self.obj.joinpath(parts).open_r()
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/util/fslike/path.py", line 136, in open_r
    return self.fsobj.open_r(self.parts)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/util/fslike/union.py", line 111, in open_r
    raise FileNotFoundError(b'/'.join(parts))
FileNotFoundError: b'terrain/15001.slp'
make: *** [Makefile:25: run] Error 1

Some facts:

I'm very surprised that the converter tells that a SLP is missing (and not a dependency for example) and that it happens during export, when I wouldn't expect it to look for files. I assumed that if the game works, then its data files must be consistent. Anyway, I'm ready to mess around to find a solution for that. If it's about the asset converter and not my installation - I have some basic C/Python skills, so with your help I am willing to make my first pull request to fix it. (Given that I manage to fix it of course...)

Looking forward to your hints!

heinezen commented 3 years ago

Hey thanks, the HD converter is actually missing a feature here. That's why it fails. The HD converter uses the AoC converter as its backbone. The file it complains about contains a terrain texture, which is stored as an SLP in AoC, but as a PNG in HD.

The fix would be to get the filename from the data and copy the file.

You can try making te fix if you want :) https://github.com/SFTtech/openage/issues/1289 has some pointers for the converter documentation. The code that creates the export requests for AoC is here:

https://github.com/SFTtech/openage/blob/ae3111a6822a91e7a6ea427f2f862a73e8458e0f/openage/convert/processor/conversion/aoc/media_subprocessor.py#L100-L111

For HD it would need to be rewritten, so that the source filename is correct. And then in the exporter there is this logic:

https://github.com/SFTtech/openage/blob/ae3111a6822a91e7a6ea427f2f862a73e8458e0f/openage/convert/processor/export/media_exporter.py#L167-L177

Here, it should just copy the file, when it ends in .png.

Also, can you show the error message for converting AoE2:DE, please? :)

wlata commented 3 years ago

Thanks for the snippets, that helped me to get some ideas. I am inspecting how the converter and the assets themselves work, it might take me a while.

And here's the output for DE. (I've got no DLCs if it plays any role) - at the moment it gives me no clue what went wrong here.

>Do you want to convert assets? [Y/n]
> y
Could not find any installation directory automatically.
Please enter an AOE2 install path manually.
> /run/media/user/7A58CF8158CF3AA1/Steam/steamapps/common/AoE2DE/
converting from '/run/media/user/7A58CF8158CF3AA1/Steam/steamapps/common/AoE2DE'
INFO [py] Compatible game edition detected:
INFO [py]  * Age of Empires 2: Definitive Edition
INFO [py] converting metadata
INFO [py] [0] palette
INFO [py] [1] empires.dat
INFO [py] Starting conversion...
INFO [py] Extracting Genie data...
INFO [py] Creating API-like objects...
INFO [py] Linking API-like objects...
INFO [py] Generating auxiliary objects...
INFO [py] Creating nyan objects...
Traceback (most recent call last):
  File "run.py", line 15, in init run
    main()
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/__main__.py", line 138, in main
    return args.entrypoint(args, cli.error)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/game/main.py", line 71, in main
    used_asset_path = convert_assets(
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/main.py", line 100, in convert_assets
    for current_item in convert(args):
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/tool/driver.py", line 30, in convert
    yield from convert_metadata(args)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/tool/driver.py", line 85, in convert_metadata
    modpacks = args.converter.convert(gamespec,
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/conversion/de2/processor.py", line 61, in convert
    modpacks = cls._post_processor(dataset)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/conversion/de2/processor.py", line 150, in _post_processor
    DE2NyanSubprocessor.convert(full_data_set)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/conversion/de2/nyan_subprocessor.py", line 38, in convert
    cls._process_game_entities(gamedata)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/conversion/de2/nyan_subprocessor.py", line 135, in _process_game_entities
    cls.unit_line_to_game_entity(unit_line)
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/conversion/de2/nyan_subprocessor.py", line 283, in unit_line_to_game_entity
    abilities_set.append(AoCAbilitySubprocessor.remove_storage_ability(unit_line))
  File "/home/user/git/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/conversion/aoc/ability_subprocessor.py", line 5175, in remove_storage_ability
    entity_ref = entity_lookups[entity.get_head_unit_id()][0]
KeyError: 1699
make: *** [Makefile:25: run] Error 1
heinezen commented 3 years ago

And here's the output for DE. (I've got no DLCs if it plays any role) - at the moment it gives me no clue what went wrong here.

Not having the DLC might play a role. Although it might be an update that's causing the issue. I'd have to check.

heinezen commented 3 years ago

The DE2 thing should be fixed in https://github.com/SFTtech/openage/pull/1395