C++ resource embedding takes too much memory while compiling with GCC

[ArcticLampyrid]

Our team is developing a C++ application using Slint, which includes embedded assets totaling around 16MiB (images, fonts, etc.). This has resulted in the Slint compiler generating a very large app.h header file (~70MiB). Compiling this under GCC 14 consumes approximately 30GiB of memory.

The excessive memory usage is causing our CI runner to fail. Given our limited resources, we are unable to allocate such a large amount of memory for CI runners.

I fully understand the inevitable inflation that occurs when converting binary files into C++ code. However, I believe there are better methods to archive it.

• Referring to the Qt framework, it provides qt_add_big_resources, which directly generates object files (.o, .obj) instead of C++ source code. This approach seems like a reasonable solution to this issue.
• Since C23, a new preprocessor #embed is provided for embedding file, which may boost the process and reduce the memory usage.

p.s.

The code of generating embed resources is at (current) https://github.com/slint-ui/slint/blob/46220259691ee498928537e7ed1d1df69b8f5bec/internal/compiler/generator/cpp.rs#L549

[tronical]

Thanks for filing this issue. I agree, the current solution doesn't scale very well.

I wish we could use #embed, but I don't think that's accessible for C++ yet?

What might be a third option that's possibly easier to implement is to generate a .o file for the data directly, using https://github.com/gimli-rs/object / https://github.com/Cr0a3/Formatic / https://crates.io/crates/formatic . This can create ELF/Mach-O/COFF (👍 ) and covers X86-64, Arm (I suppose any version), Risc-V, and WASM. Yeah, x86 is missing, but I'd say that would be an acceptable compromise.

So a way forward would be that on the CMake side we determine that we could use this approach of directly creating a .o file, then from slint_target_sources we could

1. Add an object library with add_library(${target}_slint_resources OBJECT IMPORTED GLOBAL)
2. Link that into ${target}
3. Set ${some_path}/${target}_slint_resources.o} as IMPORTED_LOCATION
4. Invoke the slint-compiler with additional options to specify object file format, target-architecture, and the path to ${target}_slint_resources.o for the compiler to generate.

[ArcticLampyrid]

Thank you for your quick response, but let's hold on a moment.

I just conducted a thorough investigation of my project and found that simply embedding resources doesn't seem to consume such a large amount of memory. Further investigation revealed that because Slint embeds the resources entirely within the header file, and our project includes this header file in multiple compilation units, the resource consumption is much greater compared to the same approach (source code generation) in Qt.

Therefore, I believe that generating a separate cpp file (as a compilation unit) to embed the resources might relieve this problem.

[ArcticLampyrid]

What might be a third option that's possibly easier to implement is to generate a .o file for the data directly, using https://github.com/gimli-rs/object / https://github.com/Cr0a3/Formatic / https://crates.io/crates/formatic . This can create ELF/Mach-O/COFF (👍 ) and covers X86-64, Arm (I suppose any version), Risc-V, and WASM. Yeah, x86 is missing, but I'd say that would be an acceptable compromise.

FYI, include_blob uses crate object to generate a .o/.obj file for the data. You may reference its code.

https://github.com/SludgePhD/include-blob/blob/2dd6add4ce5000b5291caa4af0bef5b1b52ca474/src/lib.rs#L79-L121

[tronical]

Oh, that's a great looking project! Certainly interesting for our Rust code generator.

Regarding build times, could you work around it using precompiled headers perhaps?

[ogoffart]

#embed is on track to be part of C++26 and already supported in virtually all compilers I think, so it might be worth trying to add an option to use that.

I think that's easier and more portable than trying to generate a .o

One thing that might help would also be to split the implementation and header i a .cpp file (I thought we had an issue for that already, but I can't find it)

[tronical]

Good point. We could just have a CMake "configure test for it", and if it's supported we pass some --use-cpp-embed command line option to the compiler. Almost sounds... trivial.