Failure to work with PackageCompiler due to MKL

[piever]

The following simple app fails to be executed:

module MyApp

using FFTW

function julia_main()::Cint
    @show FFTW.fftw_provider
    @show fft(rand(10)) 
    return 0
end

end # module

I'm on julia 1.7.1 with PackageCompiler v2.0.2, FFTW v1.4.5 and MKL_jll v2021.1.1+2. I'm on Windows (powershell 7.2.1), but I've noticed the issue also on other systems.

I'm creating the app with

create_app(".", "AppCompiled") # I'm  in the MyApp folder

Below is the error:

C:\Users\pietro\Documents\scratch\MyApp> .\AppCompiled\bin\MyApp.exe
  Downloaded artifact: MKL
  Downloaded artifact: MKL
fatal: error thrown and no exception handler available.
InitError(mod=:MKL_jll, error=ErrorException("Unable to automatically install 'MKL' from 'C:\Users\pietro\.julia\packages\MKL_jll\kG4RZ\Artifacts.toml'"))
error at .\error.jl:33
#ensure_artifact_installed#23 at C:\Users\pietro\.julia\juliaup\julia-1.7.1+0~x64\share\julia\stdlib\v1.7\Pkg\src\Artifacts.jl:441
ensure_artifact_installed##kw at C:\Users\pietro\.julia\juliaup\julia-1.7.1+0~x64\share\julia\stdlib\v1.7\Pkg\src\Artifacts.jl:413
unknown function (ip: 00000000177fd16c)
#ensure_artifact_installed#22 at C:\Users\pietro\.julia\juliaup\julia-1.7.1+0~x64\share\julia\stdlib\v1.7\Pkg\src\Artifacts.jl:404
unknown function (ip: 00000000177f498c)
ensure_artifact_installed##kw at C:\Users\pietro\.julia\juliaup\julia-1.7.1+0~x64\share\julia\stdlib\v1.7\Pkg\src\Artifacts.jl:399
unknown function (ip: 00000000177f4678)
_artifact_str at C:\Users\pietro\.julia\juliaup\julia-1.7.1+0~x64\share\julia\stdlib\v1.7\Artifacts\src\Artifacts.jl:547
unknown function (ip: 00000000177ecb4b)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:757
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:713
#invokelatest#2 at .\essentials.jl:716
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:713
invokelatest at .\essentials.jl:714
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:587
jl_fptr_interpret_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:675
macro expansion at C:\Users\pietro\.julia\juliaup\julia-1.7.1+0~x64\share\julia\stdlib\v1.7\Artifacts\src\Artifacts.jl:680 [inlined]
find_artifact_dir at C:\Users\pietro\.julia\packages\JLLWrappers\RYztT\src\wrapper_generators.jl:17
__init__ at C:\Users\pietro\.julia\packages\MKL_jll\kG4RZ\src\wrappers\x86_64-w64-mingw32.jl:9
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_module_run_initializer at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:73
_finish_julia_init at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:796
jl_init_with_image at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:74 [inlined]
jl_init_with_image at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:63 [inlined]
jl_init at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:90
.text at C:\Users\pietro\Documents\scratch\MyApp\AppCompiled\bin\MyApp.exe (unknown line)
__tmainCRTStartup at C:\Users\pietro\Documents\scratch\MyApp\AppCompiled\bin\MyApp.exe (unknown line)
.l_start at C:\Users\pietro\Documents\scratch\MyApp\AppCompiled\bin\MyApp.exe (unknown line)
BaseThreadInitThunk at C:\Windows\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)

I think something is off because by default I don't think FFTW should even try to use (or install) MKL to compute the Fourier transform. If I run the julia_main function normally, I get

FFTW.fftw_provider = "fftw"

It seems to be related to #210.

[Stopa42]

The problem seems to be with how lazy artifacts are loaded. This issue is also discussed at PackageCompiler#639 and the solution (which I have tested to be working) is to use include_transitive_dependencies=false in the create_app function.

[bilderbuchi]

Would turning MKL_jll into a (julia 1.9+) weak dependency be a possible solution for this problem?

[sloede]

For the record, you can either set include_transitive_dependencies=false (as referenced above) or include_lazy_artifacts=true.