Generate precompile statements

[simonbyrne]

At the moment I have jl_parse_opts(&argc, &argv); in main.c, which lets you run ./main --trace-compile=stderr to get the list of precompile statements required, which I have then manually pasted into precompile.jl.

This is probably not ideal: it requires manual intervention, and it does require main to use julia's argument parsing.

[ViralBShah]

Is it possible to use SnoopCompile?

[simonbyrne]

Hmm, I'm still getting compilation on GitHub Actions:

precompile(Tuple{Type{REPL.Terminals.TTYTerminal}, String, Base.PipeEndpoint, Base.PipeEndpoint, Base.PipeEndpoint})
precompile(Tuple{getfield(Base, Symbol("##s91#155")), Any, Any, Any, Any, Any})
precompile(Tuple{typeof(Base.merge_names), Tuple{Symbol, Symbol, Symbol}, Tuple{}})
precompile(Tuple{typeof(Base.merge_types), Tuple{Symbol, Symbol, Symbol}, Type{NamedTuple{(:tol, :maxiter, :statevars), Tuple{Float64, Int64, IterativeSolvers.CGStateVariables{Float64, Array{Float64, 1}}}}}, Type{NamedTuple{(), Tuple{}}}})
precompile(Tuple{typeof(Main.julia_cg), Ptr{Nothing}, Ptr{Float64}, Ptr{Float64}, UInt64})

Some of these are only on Julia 1.5. I'm not sure what to do about the 2nd one as it has a gensymed name.

[ViralBShah]

@kristofferC?

[simonbyrne]

I think it has something to do with merge_names.

[KristofferC]

Why not create a script that "seeds" the compilation and let PackageCompiler run it and put it in the precompile statements itself (https://julialang.github.io/PackageCompiler.jl/dev/sysimages/#tracing-1).

What's wrong with the gensymmed name?

[ViralBShah]

We weren't sure if it is ok to include it - as it felt like something internal.

[KristofferC]

It's in the docustring https://github.com/JuliaLang/PackageCompiler.jl/blob/1e611e47500e74915cd854f621929db03da34871/src/PackageCompiler.jl#L341-L345, in the manual where I linked and in an example (https://julialang.github.io/PackageCompiler.jl/dev/examples/plots/). Not sure how I can make it seem more public.

[ViralBShah]

That's pretty good. Of course, if only users actually read the documentation...

[simonbyrne]

What's wrong with the gensymmed name?

I don't know what the guarantees of consistency of gensymed names (e.g. they might not be the same across Julia versions), so we might not be able to save the list of precompiled statements.

I've also noticed that on Julia 1.5 it doesn't appear to precompile functions defined in the script (passed via the script argument) itself, e.g. I have this line: https://github.com/simonbyrne/libcg/blob/5b869e630b3535935a5b28e4d5498482586b6770/lib/precompile.jl#L2

but when I run ./main --trace-compile=stderr, it still appears in the list of statements: https://github.com/simonbyrne/libcg/runs/1131199583#step:7:10

[KristofferC]

so we might not be able to save the list of precompiled statements.

No, you would have to do the same as Julia itself which is to regenerate the list when the sysimage is rebuilt. But this is not only a problem with gensymmed functions, between Julia versions many method changes signatures gets removed, new one gets added, etc, so it's always good to re-generate if you want the best accuracy.

[KristofferC]

I've also noticed that on Julia 1.5 it doesn't appear to precompile functions defined in the script (passed via the script argument) itself, e.g. I have this line:

Main.julia_cg: I am unsure if Main is handled in a special way. I've found it best to always make sure that the stuff you want to precompile is in a package and that it gets included in the sysimage via importing it. This happens automatically if you use something like PackageCompiler.create_sysimage([:CG], ...).

[simonbyrne]

Fixed in #14