Godot seems not to export additional resources created during a custom asset import

Godot version

v4.2.beta3.mono.official [e8d57afae]

System information

Godot v4.2.beta3.mono - Windows 10.0.19045 - Vulkan (Forward+) - dedicated AMD Radeon RX 6700 XT (Advanced Micro Devices, Inc.; 31.0.14043.7000) - AMD Ryzen 9 3900X 12-Core Processor (24 Threads)

Issue description

I'm currently trying to import and render voxel models created by MagicaVoxel. Since I'm using a technique called raymarching and want to be able to easily modify the voxels at runtime, I don't generate a mesh for the voxels (i.e. marching cubes algorithm) but store them in a 3D texture and just render a simple, inverted box which acts as a volume for performing raymarching in the fragment shader.

However, to import a voxel model and be able to easily drag the generated objects into the scene, I've created a custom EditorSceneFormatImporter which will generate the scene structure and will import multiple custom VoxelBoxAsset resources which store the raw voxel data, dimensions and a few other properties. For this I've used a custom ResourceFormatLoader and ResourceFormatSaver which take care of handling the custom assets.

I've also created a custom VoxelBoxNode which gets a reference to a VoxelBoxAsset resource and creates a 3D texture during the ready event and assigns that to a shader material parameter.

I've encountered a small issue with the EditorSceneFormatImporter as it does not forward necessary parameters like the EditorImportPlugin would do so I've tried adding those parameters and added some logic from the EditorImportPlugin to the EditorSceneFormatImporter:

// EditorImportPlugin
virtual Error import(const String &p_source_file, const String &p_save_path, const HashMap<StringName, Variant> &p_options, List<String> *r_platform_variants, List<String> *r_gen_files, Variant *r_metadata = nullptr) override;

// EditorSceneFormatImporter (original)
virtual Node *import_scene(const String &p_path, uint32_t p_flags, const HashMap<StringName, Variant> &p_options, List<String> *r_missing_deps, Error *r_err);

// EditorSceneFormatImporter (modified)
virtual Node *import_scene(const String &p_source_file, const String &p_save_path, uint32_t p_flags, const HashMap<StringName, Variant> &p_options, List<String> *r_platform_variants, List<String> *r_missing_deps, List<String> *r_gen_files, Error *r_err = nullptr);

The reason why I did this was because I wanted to create additional files so I needed at least the p_save_path parameter and to also have it recognized by Godot as dependencies (appearing in the [depts] section in the .import file) I needed the gen_files.

Everything works fine in the editor now but when I export the project, the additional files are missing in the exported package which leads to the game crashin because the vbox resources are missing.

I've also tried going back to the original version and was able to fake the save path by using the source file name and generating the MD5 myself but there was no change in the resulting export so at least it can't be my customized build.

I've also tried to dig into the exporters code and I think I slowly get that the gen_files are meant to be editor only and won't be recognized by the exporter. So in the end I'm not sure whether this is a bug or if it's simply meant to be that way, one source file can only have on imported file.

This is the structure of my imported files:

MD5 file generated by godot: .godot/imported/MyAsset.vox-57a4ea2cd2339ea15b1e3626054167c3.md5
Scene file imported by my custom importer: .godot/imported/MyAsset.vox-57a4ea2cd2339ea15b1e3626054167c3.scn
Voxel box #1 (like a tree trunk): .godot/imported/MyAsset.vox-57a4ea2cd2339ea15b1e3626054167c3_c6294cac9bc54e6d1ed056f32fdfe48f.vbox
Voxel box #2 (like a tree crown): .godot/imported/MyAsset.vox-57a4ea2cd2339ea15b1e3626054167c3_c6294cac9bc54e6d1ed056f32fdfe48f.vbox

I was used to import assets in unity that way, you can have several imported files from one source file like a 3D model which imports a prefab (scene in Godots terms), meshes, animations, textures, materials etc. and all those imported files will be exported at the end.

I will try another workaround now where I just create a directory with the additional, generated resources right beside the file to import like MyAsset.vox will become:

MD5 file generated by godot: .godot/imported/MyAsset.vox-57a4ea2cd2339ea15b1e3626054167c3.md5
Scene file imported by my custom importer: .godot/imported/MyAsset.vox-57a4ea2cd2339ea15b1e3626054167c3.scn
Voxel box #1 (like a tree trunk): Models/MyAsset/MyAsset-c6294cac9bc54e6d1ed056f32fdfe48f.vbox
Voxel box #2 (like a tree crown): Models/MyAsset/MyAsset-c6294cac9bc54e6d1ed056f32fdfe48f.vbox

Steps to reproduce

Create a new Godot project
Create an addon with a custom importer
Import multiple files instead of just one to .godot/imported
Export the project, extract the PCK and see that it's not included in the export
Also note that one is not able to select the additional imported file via the UI

Minimal reproduction project

N/A

Importing the files to another path outside from the .godot/imported directory also did not lead to them being exported, those files are just not recognized by the exporter.

However, I found another solution where I import all additional files to res://.GenFiles/{HASH_OF_SOURCE_FILE}/{HASH_OF_GENERATED_FILE}.vbox in tandem with an EditorExportPlugin which adds all files in res://.GenFiles to the export, like this:

[Tool]
public partial class VoxelBoxExportPlugin : EditorExportPlugin
{
    public override void _ExportBegin(string[] features, bool isDebug, string path, uint flags)
    {
        var genFilesPath = $"res://.GenFiles";
        var contents = Directory.EnumerateFiles(ProjectSettings.GlobalizePath(genFilesPath), "*.*", SearchOption.AllDirectories);

        using var dirAccess = DirAccess.Open(genFilesPath);
        dirAccess.IncludeHidden = true;

        foreach (var fileName in contents)
        {
            var index = fileName.IndexOf(".GenFiles", StringComparison.InvariantCulture);
            if (index < 0)
            {
                continue;
            }

            var resFilePath = $"res://{fileName.Substring(index).Replace("\\", "/")}";

            GD.Print($"Adding custom file: {resFilePath}");

            using var fileAccess = FileAccess.Open(resFilePath, FileAccess.ModeFlags.Read);
            var data = fileAccess.GetBuffer((uint)fileAccess.GetLength());

            AddFile(resFilePath, data, false);
        }

    }
}

This workaround is fine for me now so I can finally start working on the actual game. I would still love to see a feature where additional files are automatically included in an export - in theory, the exporter could go through the scene graph and inspect which resources are references by custom nodes and automatically include them in the build.

public partial class MyNode : Node3D {
  [Export]
  private MyResource resource;
}

In the example above it should be clear that an instance of MyNode will break if the reference to a resource MyResource is not available in an export just because the file was not exported.

godotengine / godot