search

grin-compiler / ghc-whole-program-compiler-project

GHC Whole Program Compiler and External STG IR tooling
116 stars 12 forks source link

Feature request: have a way to combine all of the .o_stgbin files into a single .o_stgbin #2

Closed ckoparkar closed 3 years ago

ckoparkar commented 4 years ago

Right now, ghc-wpc generates a single file containing the external-STG (.o_stgbin) for each module. And gen-exe knows how to read a .ghc_stgapp file to combine all the .o_stgbins that make up a program. But it would be useful to have something that lets a user say "spit out all the STG for this program into a single file". Essentially, this will move the "piece together a whole program" part out of gen-exe into it's own thing.

This would be useful for other whole-program compilers that want to use external-STG as a frontend -- they would have to read a single source file to compile it. We'd like to experiment with this Idea in Gibbon for example.

csabahruska commented 4 years ago

Should this be .o_stgbin or any other file format? I guess what you need is a single self contained file, with convenient tooling (i.e. pretty printer) and library functions (load everything into data structure).

ckoparkar commented 4 years ago

I guess what you need is a single self contained file, with convenient tooling (i.e. pretty printer) and library functions (load everything into data structure).

Oh, yeah. It could be in any format, serialized or plain-text.

isovector commented 4 years ago

How much effort would this require? I'm interested in writing a haskell transpiler, and the transitive .o_stgbin seems like everything I'd need to do so.

csabahruska commented 4 years ago

In July I started to work on this feature, just have not finished, but I'll do it then. I'll add an application container format (.app_pak, a .zip file with custom file extension) that will contain the dependencies of the whole application transitively. I'll provide the tooling and library with the .app_pak handling API. It is not complicated to implement it, but I'll need to make a new ghc-wpc binary release. I'll need this feature also for the external-stg interpreter.

What is your compilation target language/platform?

isovector commented 4 years ago

Currently planning on targeting lua5.2. I'm going to spend the weekend writing an STG interpreter and see if I can get a single self-contained module up and running.

csabahruska commented 4 years ago

Sounds interesting. I'd be curious how haskell would perform with luajit. Regarding the STG interpreter, did you develop your own STG AST?

csabahruska commented 3 years ago

Finally I did it. Now GHC-WPC generates the APP_NAME.ghc_stgapp file which is just a text based project description file that contains information about the project modules and location. Then the whole haskell program IR can be packed into a single .fullpak (zip) file with the mkfullpak (GHC-WPC) cli tool. Then there is the getFullpakModules utility function in Stg.Program module to load all program modules:

getFullpakModules :: FilePath -> IO [Module]
csabahruska commented 3 years ago

Sample .fullpak file. Gunzip it before use. minigame.fullpak.gz

ckoparkar commented 3 years ago

Super! Thanks @csabahruska. I'll try it out soon.