search

tshort / StaticCompiler.jl

Compiles Julia code to a standalone library (experimental)
Other
500 stars 30 forks source link

GPUCompiler rewrite #43

Closed tshort closed 2 years ago

tshort commented 3 years ago

This is a general issue to discuss plans on how to rewrite using GPUCompiler. Now that https://github.com/JuliaLang/julia/pull/38642 is merged, it'll be easier for people to try out static compilation.

cc @jpsamaroo, @el-oso

tshort commented 3 years ago

I'll start on some housekeeping things like closing most of the PR's and issues.

jpsamaroo commented 3 years ago

I suspect you should still keep a lot of logic, but notably irgen, opt, and mcgen are replaceable by GPUCompiler.

tshort commented 3 years ago

I ditched all the existing code in #46.

One part I don't get is handling globals (string, symbols, and many others). I see placeholders for them. We need some way to populate those. I don't know if the code in libjulia can do that.

jpsamaroo commented 3 years ago

Where do see those placeholders? I couldn't find them when I looked for them.

tshort commented 3 years ago

I rechecked. I was wrong on that--must have misread it.

jpsamaroo commented 3 years ago

So I think the issue we'll run into is Julia's weird way of emitting globals, which sets them null with internal linkage. This already causes issues with GPUCompiler's optim pipeline, which does GlobalOpt and GlobalDCE, which causes such globals to be deleted and the null value inlined to loads of the global.

dumblob commented 3 years ago

This may sound rude, but I'll ask anyway :wink:. What's the current state of this?

(I don't follow Julia development, so feel free to elaborate also about that as it seems tightly related)

tshort commented 3 years ago

The current state is stagnant. I don't know that anyone's doing much right now. The path forward (I think) involves finishing https://github.com/JuliaGPU/GPUCompiler.jl/pull/125 and getting it to work with the latest version of LLVM.

The ongoing work on Julia compiler plugins (https://github.com/JuliaLang/julia/pull/41632 and https://github.com/JuliaCompilerPlugins/) is also interesting. The ability to customize code generation may simplify GPUCompiler.

jpsamaroo commented 3 years ago

I agree with @tshort that Compiler Plugins will probably be the way to go for a purely static, no runtime approach. For most users, however, I think that https://github.com/JuliaLang/julia/pull/41936 + PackageCompiler + "tree shaking" will do most of what we want. The sysimage generation system already does a lot of what we're trying to do here, and is very well tested (since every Julia user relies on it). https://github.com/JuliaLang/julia/pull/40414 will make PC binaries much faster to generate too, making it a more enjoyable user experience.

dumblob commented 3 years ago

Thanks a lot for prompt pointers @tshort and @jpsamaroo . The links provide promising information - I'll follow them and see what'll happen as it seems to be a hot topic recently.

MasonProtter commented 2 years ago

Okay to close this issue? :grinning: