Image::compress / compress_with_channels unreliable, not exportable

[TokisanGames]

Godot version

Godot 4.1.1-stable

System information

Windows 11/64, RTX 3070, Vulkan

Issue description

I'm building Terrain3D and ran into problems with Image::compress() and Image::compress_from_channels() which don't work on export and are unreliable in the editor.

#define COLOR_RB Color(1.0f, 0.0f, 1.0f, 1.0f)
#define COLOR_NORMAL Color(0.5f, 0.5f, 1.0f, 1.0f)

Ref<Image> Terrain3DStorage::get_filled_image(Vector2i p_size, Color p_color, bool p_create_mipmaps, Image::Format p_format) {
    Ref<Image> img = Image::create(p_size.x, p_size.y, p_create_mipmaps, p_format);
    img->fill(p_color);
    if (p_create_mipmaps) {
        img->generate_mipmaps();
    }
    return img;
}

img = get_filled_image(albedo_size, COLOR_RB, true, Image::FORMAT_RGBA8);
img->compress(Image::COMPRESS_S3TC, Image::COMPRESS_SOURCE_SRGB);
// img->get_format() 
// In editor: returns 19 FORMAT_DXT5, expected
// On export: returns 5 FORMAT_RGBA8, not expected **

img = get_filled_image(normal_size, COLOR_NORMAL, true, Image::FORMAT_RGBA8);
img->compress(Image::COMPRESS_S3TC, Image::COMPRESS_SOURCE_SRGB);
// img->get_format()
// In editor: returns 17 FORMAT_DXT1, not expected **
// On export: returns 5 FORMAT_RGBA8, not expected **

We transitioned to

img->compress_from_channels(Image::COMPRESS_S3TC, Image::USED_CHANNELS_RGBA);
// In editor: 19 FORMAT_DXT5, expected
// On export: returns 5 FORMAT_RGBA8, not expected **

img->compress_from_channels(Image::COMPRESS_S3TC, Image::USED_CHANNELS_RGBA);
// In editor: 19 FORMAT_DXT5, expected
// On export: returns 5 FORMAT_RGBA8, not expected **

With either function, debug exports give this error message:

ERROR: Condition "!_image_compress_bc_func" is true. Returning: ERR_UNAVAILABLE
   at: compress_from_channels (core/io/image.cpp:2637)

I'm using the default export options:

Godot 4.1.1-stable Terrain3D 0.8.1+ a69eff

Steps to reproduce

See above

Minimal reproduction project

n/a

[clayjohn]

Looks like most of the image compression/decompression libraries aren't included in export builds.

In the past I know that we avoided including any image saving/loading libraries in the export builds, but I know that has been slowly opening up.

I guess the discussion here should be whether we want to enable the compression libraries for export builds. It would be helpful to do a build size analysis to see what the realistic cost for enabling them is.

[Calinou]

I guess the discussion here should be whether we want to enable the compression libraries for export builds. It would be helpful to do a build size analysis to see what the realistic cost for enabling them is.

I'd suggest adding build-time options to enable them in export templates when needed, similar to https://github.com/godotengine/godot/pull/73003.

[TokisanGames]

I now understand that the function doesn't work because it's not even there, by design.

We're using these functions when the user creates terrain materials with null textures. We want to create blank textures automatically, but now we cannot. I will have to work around this by putting warnings everywhere that the user cannot leave it with null textures. It's a bit tacky, and users will ignore the warnings and then complain about why their exports don't work.

Philosophically, I don't understand why there would be an API for us to use, yet a random portion of the API is removed on export. If the engine runs itself as a game and has access to all of these functions and libraries, my project should too. I don't like having to spend time troubleshooting a broken project only to find it's one of those neutered libraries taken out just because.

I like the idea that on the export screen we have the option of including or excluding every possible library that we have access to in the editor.

Build time options, would allow us to put that in the instructions and include an export_template.cfg.

[Calinou]

We want to create blank textures automatically, but now we cannot.

Can't you store premade compressed textures in the add-on, make sure these are exported and use those if the user hasn't specified any?

Philosophically, I don't understand why there would be an API for us to use, yet a random portion of the API is removed on export.

That's what documentation is for :slightly_smiling_face:

I like the idea that on the export screen we have the option of including or excluding every possible library that we have access to in the editor.

This is not feasible without recompiling export templates. (This is how https://github.com/godotengine/godot/pull/62996 works already; it won't work if you don't compile custom export templates with the build profile used at build-time.)

We don't want to require having a build toolchain set up to export projects, so this can't be provided by default. Build instructions also vary heavily depending on platform and cross-compilation isn't always possible.

[TokisanGames]

Can't you store premade compressed textures in the add-on, make sure these are exported and use those if the user hasn't specified any?

We are creating image blanks -> textures -> texture array. All of the images have to be the same size to go in the array, so if the user has already put in a 1k x 1k texture, we need to match that. We'd have to include multiple sizes from 256, which some are using, all the way up to 4k just for blanks, *2 because we need a different type for normal.

We could exclude the material from the array, but then our array indexing on the CPU will be different from the shader. 🤔 I'll probably just leave warnings everywhere for now if they have empty materials until we have the option to include image compression.

[TokisanGames]

I've found a solution for us. Rather than generate the image blanks at runtime, which doesn't work on export. Now I generate the texture blanks while in the editor and save it with the rest of the data.

Do you guys want to leave this open to update the documentation or provide the option of including the compression libraries on export?