Plots.jl not precompiling with julia-py

[manuelbb-upb]

I am working on a package that has 'Plots.jl' as one of its dependencies. I test and use the package in a Singularity container based on Ubuntu 18.04. Because of its long startup time I want to create a custom sysimage to use with JuliaPy. I have managed to create a sysimage for my package by removing the plotting functionality and dependencies.

If I include Plots, I get an error. The error can be reproduced by simply starting julia-py and using Pkg.add(Plots) and using Plots. This results in

ERROR: InitError: could not load library "/opt/julia_pkgs/artifacts/e6e5f41352118bbeb44677765ebccab8c151c72a/lib/libssl.so"
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_0i' not found (required by /opt/julia_pkgs/artifacts/e6e5f41352118bbeb44677765ebccab8c151c72a/lib/libssl.so)
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Libdl/src/Libdl.jl:109
 [2] dlopen at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at /opt/julia_pkgs/packages/OpenSSL_jll/2nwam/src/wrappers/x86_64-linux-gnu.jl:88
 [4] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:781
 [6] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:712
 [7] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:770
 [8] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:712
 [9] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:770
 [10] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:712
 [11] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:770
 [12] _require(::Base.PkgId) at ./loading.jl:1006
 [13] require(::Base.PkgId) at ./loading.jl:927
 [14] require(::Module, ::Symbol) at ./loading.jl:922
during initialization of module OpenSSL_jll 

Above I used a custom DEPOT_PATH, but this happens with the default settings and in a fresh environment too. The error does not occur using julia instead of julia-py. Funny enough the error is not thrown if I issue using Plots a second time.

As I do not want to maintain separate versions of my package (with and without Plots) and the sysimage compilation is meant to be automated, my current workaround is two-fold:

1) I use the following script (patch_openssl.jl) to get the artifact path indicated by the error message.

using Pkg
using Pkg.Artifacts

envpath = tempname()
mkdir(envpath)
Pkg.activate( envpath )

Pkg.add( "OpenSSL_jll" )

hash_val = artifact_hash("OpenSSL", joinpath( dirname( Base.find_package("OpenSSL_jll") ), "..", "Artifacts.toml" ) )
lib_path = joinpath( artifact_path(hash_val), "lib" )

Pkg.activate()
rm( envpath, recursive = true, force = true)

println(lib_path)

The error seems to happen because somehow the system's version of OpenSSL is picked over the version shipped with the julia package. 2) I then prepend the result to the environment variable LD_LIBRARY_PATH (outside of julia):

export LD_LIBRARY_PATH=$(julia patch_openssl.jl):$LD_LIBRARY_PATH

Simply changing ENV["LD_LIBRARY_PATH"] in the REPL and then trying to use Plots does not work.

[manuelbb-upb]

Seems to work no without any problem using Julia 1.5 and version 0.5.3 of pyjulia. My current workflow (without any modifications to LD_LIBRARY_PATH) utilizes the new options offered by python -m julia.sysimage.

1. First I create a new sysimage of my module using PackageCompiler, say "my_base_img.so".
2. I then run
   python -m julia.sysimage final_img.so --base-sysimage my_base_img.so which certainly takes a while but throws no more errors.

Thus I would consider this issue closed.