bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.13k stars 4.05k forks source link

Embedding Resources into Binary (Resource Compiler) #13577

Open cpsauer opened 3 years ago

cpsauer commented 3 years ago

Hi wonderful Bazel folks, this one's a feature request.

It'd be awesome to be able to embed resource files in native binaries (across platforms). This would be great for having resources (like images, model/data files, etc) easily portable within binaries without always needing to do platform specific resource handling.

It strikes me that Bazel is uniquely suited to solve this, with its easy ability to generate code to be compiled into the dependency graph. It seems like the easiest way to do it would be to generate some source files (or an object file directly) containing the bytes of the resource file and then link that in, accessible at a given symbol. Here's an article that sketches that kind of thing out, including how it's done for cmake--though you probably want to be careful to use extern here to keep from blowing up the build artifacts.

Thanks for your consideration, Chris (ex-Googler)

P.S. Related: There's an older Windows-specific issue here https://github.com/bazelbuild/bazel/issues/8273--windows tooling also referred to in that article I linked.

cpsauer commented 2 years ago

Noting down some outside tools that might be interesting/possible to wrap into bazel rules: https://github.com/mhekkel/mrc (though it doesn't currently work on Apple/mach-O -> Probably a source-generating approach would be better.) https://github.com/markusfisch/cpprc The xxd tool with the -i option. (though doing this with headers is clunky and duplicative)

Probably better to just write a quick (Python) tool that dumps files -> C/C++ header&source and compile those, bootstrapping cross-platform support off of other tools. Bazel's magic should make that quite easy! Would make sense to access through symbols instead of string keys (as in some previous options), so we can take advantage of the linker in, e.g., optimizing out unused resources. That


embed (or std::embed) may someday build support for this into the language. See https://thephd.dev/finally-embed-in-c23 and https://clang.llvm.org/c_status.html for status. Learned about that from seeing progress on related things over at boringssl

Those articles also point out the slowness inherent in the convert & expand -> parse process in, e.g., the python tool version above. The long term solution should probably be based on #embed


Another approach--probably the best in the meantime--is to wrap platform-specific implementations. One could use ld, as here: https://stackoverflow.com/questions/4158900/embedding-resources-in-executable-using-gcc And use the Windows resource compiler on Windows, as in #8273

github-actions[bot] commented 3 months ago

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

cpsauer commented 3 months ago

activity (since there was some on the Windows's sub issue)

Looks like clang 19 has partial support now for #embed, btw.