Full project export

[nikitalita]

This adds the following features:

• Full project export
• Support for exporting the project.godot file
• Support for texture conversion for godot v2, v3, and v4 textures.
• Converting binary resources to text resources without loading.
• Converting mp3str to mp3
• Rewriting the metadata on v2 imported resources to point to newly converted assets

Full Project Export:

This is currently only available via cli option. It will take in a PCK and extract all the files, decompile all the scripts, and convert all the imported assets it has support for. This works on v2, v3, and v4.

This can be done with either the editor binary:

.\bin\godot.windows.tools.64.exe --no-window --path modules\gdsdecomp\standalone --extract=<PCK_FILE> --output-dir=<OUTPUT_DIR>

or the standalone binary:

GDRE_Tools.exe --no-window --extract=<PCK_FILE> --output-dir=<OUTPUT_DIR>

It writes a log file to the output directory, and informs the user:

• if any imported assets failed to be converted,
• any assets were converted from a lossy import, which may lead to loss of fidelity when reimported
• any assets saved in such a manner that it won't be reimported when opening the project
• any assets that we don't currently have support for converting

Relevant source files:

• gdre_main.gd - the cli parsing and entry point for the main functions are here
• gdre_packed_data.cpp - pack opening and loading logic
• gdre_settings.cpp - keeps track of any loaded pack files and sets certain godot internals to make the loaded pack paths globally accessible via FileAccess() like in a normal packed executable. It also adds the logger to the Godot composite logger.
• gdre_cli_main.cpp - currently just holds the GDRESettings singleton, but most of the logic in gdre_main.gd will eventually be moved here
• pck_dumper.cpp - handles verifying the checksum of the pack files and extracting them
• import_exporter.cpp - this will read all the *.import files, load the relevant data, and use that to convert the imported assets.

Support for exporting the project.godot file

This is available in the editor when extracting a pck, or via CLI export. Supports godot v2, v3, and v4. Relevant files:

• pcfg_loader.cpp

Support for texture conversion for godot v2, v3, and v4 textures.

This is available in the editor, or via CLI export. Supports godot v2, v3, and v4.

Relevant files:

• image_parser_v2.cpp - Converting the old v2 Image Variants to the Image objects
• texture_loader_compat.cpp - Loads the v2, v3, and v4 texture files as v4 StreamTextures.

Converting binary resources to text resources without loading.

This is a available in the editor, or via CLI export. Supports godot v2, v3, and v4 This was made for both compatibility with v2, v3, and v4 assets, and to avoid loading assets which may or may not exist in the pak. In the Godot editor, you can only currently convert a resource to text after fully loading it, which may fail due to this. This also meant that any external resources that the resource pointed to which didn't exist would be removed entirely upon saving it as a text resource. This also enables other things, like support for converting v2 textures (see above).

relevant files:

• resource_loader_compat.cpp

Converting mp3str to mp3

This is a available in the editor, or via CLI export. Supports godot v3, and v4 This was pretty simple to add; Godot 3.3 will have support for this, and it's already on the master and 3.x branches and it's already making its way into games.

Rewriting the metadata on v2 imported resources to point to newly converted assets

This only happens during CLI export.

This was necessary for v2 project export because of how Godot 2.x handles imports. It allows for assets outside of the project tree to be imported, and it writes the import metadata to the imported resources themselves. This means that the assets will be permanently pointing to a location that is likely out of the project tree. with no real way to re-import them except editing them one by one, manually. We fix this by putting the converted resources under ".assets" in the exported project folder and re-saving the imports with the new source locations in the metadata.

relevant files:

• resource_loader_compat.cpp
• resource_import_metadatav2.cpp

Other considerations

This is a pretty huge PR; I tried to limit the edits to the existing files (like the editor, which only has minor edits to use the new loaders). If necessary, I can try and break this up but it might take a while.

This is based on the branch with my previous PR, but they're out of sequence, so I may have to rebase once that's merged.

[bruvzg]

Seems like it does not work. Version detection always fail, and no files are extracted and converted:

Successfully loaded PCK!
Version:
No errors detected!
Unknown version, no decomp
Number of import files: 0

Tested with 3.2 stable, 4.x current and 3.x current PCKs.

Probably something wrong with the PCK loading.

[nikitalita]

Just did a quick test; absolute paths work, while relative paths do not. The problem is that godot will change the directory to wherever the standalone files are stored before running, so any paths will be relative to that directory, not the cwd where you actually ran the command. So, even if I do get_current_dir() at the beginning, I can't know where the actual cwd was. I'm not sure if it's possible to recover that with the way that Godot is set up, either.

In the meantime, try testing it with absolute paths.

[bruvzg]

The problem is that godot will change the directory to wherever the standalone files are stored before running, so any paths will be relative to that directory, not the cwd where you actually ran the command. So, even if I do get_current_dir() at the beginning, I can't know where the actual cwd was. I'm not sure if it's possible to recover that with the way that Godot is set up, either.

Seems to work fine with absolute paths. This issue probably can be fixed by adding custom Error set_cwd(const String &p_cwd) to the subclassed OS.