JuliaLang / PackageCompiler.jl

Compile your Julia Package
https://julialang.github.io/PackageCompiler.jl/dev/
MIT License
1.42k stars 190 forks source link

Precompile Script [extras] for Apps #689

Open bradcarman opened 2 years ago

bradcarman commented 2 years ago

Can one build an app of a package which has an [extras] section? Meaning, I have a test script for my package/app which uses external dependencies (i.e. Test, ReferenceTests.jl, etc) that I would like to use for the precompile script.

For my current project, I get an error that "ReferenceTests is not found in the current path", but I've added this dependency to the [extras] section with the [targets] test = ["Test", "ReferenceTests", etc..]. Should this work for precompile scripts when building apps?

bradcarman commented 2 years ago

To make my question more clear, I've added a MWE here: https://github.com/bradcarman/ExampleApp

When attempting to run the App/test/runtests.jl as the precompile_execution_file I get the following error...

julia> create_app("App", "AppCompiled"; precompile_execution_file="App/test/runtests.jl", incremental=true, force=true)
┌ Warning: The project dependencies or compat requirements have changed since the manifest was last resolved.
│ It is recommended to `Pkg.resolve()` or consider `Pkg.update()` if necessary.
└ @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.8\Pkg\src\API.jl:1527
┌ Warning: The project dependencies or compat requirements have changed since the manifest was last resolved.
│ It is recommended to `Pkg.resolve()` or consider `Pkg.update()` if necessary.
└ @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.8\Pkg\src\API.jl:1527
┌ Warning: The project dependencies or compat requirements have changed since the manifest was last resolved.
│ It is recommended to `Pkg.resolve()` or consider `Pkg.update()` if necessary.
└ @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.8\Pkg\src\API.jl:1527
[ Info: PackageCompiler: Executing C:\Work\IST\Projects\JuliaPackages\ExampleApp\App\test\runtests.jl => C:\Users\CarmanBr\AppData\Local\Temp\jl_packagecompiler_qYW6vS\jl_B518.tmp
ERROR: LoadError: ArgumentError: Package ReferenceTests not found in current path.
bradcarman commented 2 years ago

OK, I see what the problem/solution is. I've implemented the following on a fork...

function run_precompilation_script(project::String, sysimg::String, precompile_file::Union{String, Nothing}, precompile_dir::String; extras_project::Union{String,Nothing}=nothing)
    tracefile, io = mktemp(precompile_dir; cleanup=false)
    close(io)
    arg = precompile_file === nothing ? `-e ''` : `$precompile_file`
    cmd = `$(get_julia_cmd()) --sysimage=$(sysimg) --compile=all --trace-compile=$tracefile $arg`
    # --project is not propagated well with Distributed, so use environment
    splitter = Sys.iswindows() ? ';' : ':'
    envs = [project]
    !isnothing(extras_project) && push!(envs, extras_project)
    push!(envs, "@stdlib")
    cmd = addenv(cmd, "JULIA_LOAD_PATH" => join(envs, splitter))
    precompile_file === nothing || @info "PackageCompiler: Executing $(abspath(precompile_file)) => $(tracefile)"
    run(cmd)  # `Run` this command so that we'll display stdout from the user's script.
    precompile_file === nothing || @info "PackageCompiler: Done"
    return tracefile
end

I propagate the extras_project keyword and now I can run...

julia> create_app("App", "AppCompiled"; precompile_execution_file="App/test/runtests.jl", extras_project="@v1.8")

Now I can have ReferenceTests.jl in my general @v1.8 environment and use my test script to compile my App.

Should I submit a Pull Request for such a feature?