Complete Compilation: `autosnoop`? `--compile=all`? Disable compilation at runtime?

[NHDaly]

Figure out how to really, actually compile everything at build-time and do no compilation at runtime!!

There was some discussion in-person at JuliaCon around the fact that things aren't being compiled fully because we're only building with --compile=yes instead of --compile=all. (During the talk I incorrectly said that we were building with --compile=all.)

Therefore, the belief is that julia is only precompiling everything (parsing and lowering) rather than compiling to machine code. Some possible fixes we've discussed:

• Use a snoopfile to force compilation.
  • I added autosnoop the day before my talk, which just creates a snoopfile with the contents julia_main([""]), causing the program to be executed during compilation, building a snoopfile, and compiling with it.
  • @JeffBezanson suggested offline, that maybe autosnoop should simply add the following line to your program: precompile(julia_main, (Array{String, 1},))
• ACTUALLY set --compile=all
• Set --compile=none at runtime (i guess this would be passed via the C api's jl_options to julia_init).

[JeffBezanson]

If the JIT will still be available at runtime (which it probably will be in most cases), a good option is --compile=min, which will compile only when necessary. Otherwise there can be code that's un-runnable since it requires the compiler (e.g. code using llvmcall).

[NHDaly]

Thanks Jeff, that's excellent. I'm playing around with it now!

[NHDaly]

maybe autosnoop should simply add the following line to your program: precompile(julia_main, (Array{String, 1},))

Unfortunately, it doesn't look like this simple precompile trick will work.. If I generalize correctly from this example, it looks like precompile isn't recursive. That is, precompiling julia_main won't cause it to precompile your entire program:

julia0.7> foo(x) = 3 + x - 2 + x*x + x^2 + 2 - 1 / x
>> foo (generic function with 1 method)

julia0.7> bar(x) = foo(x) + x
>> bar (generic function with 1 method)

julia0.7> @time precompile(bar, (Int64,))      # First time is slow, actually compiling.
  0.021747 seconds (25.83 k allocations: 1.385 MiB, 33.75% gc time)
>> true

julia0.7> @time precompile(bar, (Int64,))      # This time is fast.
  0.000013 seconds (7 allocations: 320 bytes)
>> true

julia0.7> @time precompile(foo, (Int64,))      # First time is slow again, which means `foo` didn't get compiled from `precompile(bar)`
  0.003393 seconds (77 allocations: 5.063 KiB)
>> true

julia0.7> @time precompile(foo, (Int64,))      # This time is fast.
  0.000015 seconds (7 allocations: 320 bytes)
>> true

So probably if the solution ends up taking an autosnoop shape rather than a compile=all shape, it'll have to actually run the program to generate the snoopfile.

[NHDaly]

Cool!!

So just simply adding compile=all at build-time, and compile=no at run-time, I was able to get a fully statically compiled binary working! And it did, indeed, start rendering its plot immediately, because we get skip compiling Plots at runtime! :D

So I can confirm that it is compiling everything ahead of time with compile=all. :)

NOTE: Just doing compile=all without the runtime compile=no, seems to behave exactly the same as before, which I think points towards it recompiling at runtime for your native machine. The first plot is still slow, and the later plots are fast.

However, interestingly, with compile=no, while the first plot is instantaneous, actually rendering the plots is slower... I'm guessing this is because without runtime compilation it's having to fall back to a whole bunch of type unstable functions? Weirdly, though, compile=min doesn't seem to be any better...

[NHDaly]

Haha damn, these apps are half a Gig each... yeesh. What's Electron 350MB for??

Anyway, i've uploaded them here for anyone following along, if you want to compare them yourself! :) https://github.com/NHDaly/jupyter-binder/releases/download/ApplicationBuilder-dumps/SinePlotterBundled-compile-min.zip https://github.com/NHDaly/jupyter-binder/releases/download/ApplicationBuilder-dumps/SinePlotterBundled-compile-no.zip