terralang / terra

Terra is a low-level system programming language that is embedded in and meta-programmed by the Lua programming language.
terralang.org
Other
2.72k stars 201 forks source link

Embed internal files without geninternalizedfiles.lua #413

Open ErikMcClure opened 5 years ago

ErikMcClure commented 5 years ago

Once #412 is merged, the only remaining external lua script being run will be geninternalizedfiles.lua, which just creates a header file full of gigantic byte arrays of a bunch of files used by Terra. This is a silly thing to do when there are standard ways of including read-only data inside executables. Windows has resource files, and Linux has this method of embedding resources in the .rodata section.

However, both of these techniques are fully general - they can embed any kind of binary data, but all of the data Terra is embedding are just text strings. There's no errant null terminators inside of them (I checked), which means they could just be embedded as strings inside the code. It is possible that doing this might not guarantee then end up in the correct .rodata section, but I am reasonably certain it is possible to interpret an entire header file as a raw string by doing some absurd preprocessor shenanigans.

The only tricky part about this is that Terra uses this mechanism to absorb the entire contents of the clang resource directory. This means the build step would still have to construct, at minimum, some sort of simplified header file that contains the logic to include the files, without the complex machinations of geninternalizedfiles.lua.

If successful, this would eliminate the only remaining external lua script in the build process, which will hopefully reduce the chances of it failing for strange reasons.