Open eRedekopp opened 1 year ago
I have narrowed down the issue to StockFlow specifically. I am discussing with the package's maintainers. Any added information would be greatly appreciated
It seems to happen with other packages too. See below
I also get a process that never seems to finish, when compiling the code at https://github.com/mcreel/Econometrics, using the MakeSysImage.jl script that is part of the code. With 1.8.5, it finishes in about 10 minutes.
Thanks! I tried it with 1.6.0 and it compiled in a few minutes also. This seems to be an issue with version 2 specifically
I'm just here to write that I'm having the same problems (with Julia 1.9.0).
I've had the same problem with even a simpler file, just following the example in https://julialang.github.io/PackageCompiler.jl/dev/examples/plots.html (I've documented it in https://discourse.julialang.org/t/sysimage-creation-example-with-plots-does-not-seem-to-work/101269).
But an even simpler example also seems to spin forever:
julia --history-file=no --startup-file=no
using PackageCompiler
create_sysimage(:Plots, sysimage_path="sys_plots.so")
Since this issue can be seen just with Plots, it is not necessarily due to StockFlow.
On the other hand, I can create sysimages for other packages; for example I can successfully reproduce the example in https://julialang.github.io/PackageCompiler.jl/dev/examples/ohmyrepl.html and I can also create sysimages that contain other packages, for example this works:
using PackageCompiler
create_sysimage([:LanguageServer, :SymbolServer];
sysimage_path="sys_languagesymbolserver.so")
Agreed -- I don't think this has to do with StockFlow specifically. I've edited my previous comment. I've played around with this quite a bit more and am having trouble finding patterns in when it does or does not work.
I ended up giving up on my original plans for this a couple weeks ago; instead I created a basic HTTP server that constantly runs and has everything pre-compiled, although this is not my preferred solution.
In my case, removing all calls to Plots (or StatsPlots) seems to allow the creation of the sysimages I use. So that is what I am doing for now.
@rdiaz02 yea that worked for me too. Not sure why.
I'm not sure I understand that as a workaround. If we remove the calls to Plots from the sample script, doesn't that mean that the functions won't be pre-compiled, and so the only speedups will be in the 'using Plots' call and nothing else? In any case, I don't see how that would fix it for the minimal reproducible example in the original post
@eRedekopp You are correct and this is not really a workaround, because of what you say. But, in the meantime, while this bug is fixed, removing Plots at least allows the creation of sysimages that use (at least some) other packages.
Understood. Thanks for the clarification
Hitting the same after updating 1.8.5 to 1.9.2, also with Plots.jl in the Project. Another Project without Plots.jl works. Any ideas why @KristofferC?
With Julia 1.10rc2, a project that includes Plots.jl successfully compiles faster than ever before! However, on the first try, it errors, and says to call Base.retry_load_extensions()
. After doing this, on the second try, the compilation is successful. The specific script I used is https://github.com/mcreel/Econometrics/blob/main/MakeSysimage.jl, for a project with many dependencies, those in https://github.com/mcreel/Econometrics/blob/main/Project.toml
I'm also having the issue with Plots.jl that @rdiaz02 described. Would be great to get a fix in Julia 1.9 out for this if possible? @KristofferC
It sounds like (for the Plots.jl problem), that Julia 1.9.0 is broken and Julia 1.10.0-rc2 is fixed?
If this is correct, then what would be really helpful is if someone could do a git bisect
between Julia 1.9.0 and Julia 1.10.0-rc2 to figure out which PR fixed the issue. Then, the (usually) easy part is backporting that PR to Julia 1.9.x.
I'm here to confirm this issue, too. The source code points to the C compiler call. The C compiler seems to not return. I tried different C compilers via JULIA_CC
, but no changes.
I am trying to compile a sysimage using PackageCompiler but have been unable to successfully complete the process. Everything appears to work correctly until I reach "PackageCompiler: compiling incremental system image", at which point it spins forever. I left it overnight and had it running at least 11 hours before killing the process.
When I kill the process, it always gives me the following stacktrace, which appears to show the process stuck in a spinlock.
Julia Info
Julia Version 1.9.1 Commit 147bdf428cd (2023-06-07 08:27 UTC) Platform Info: OS: Linux (x86_64-linux-gnu) CPU: 2 × AMD EPYC 7B12 WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-14.0.6 (ORCJIT, znver2) Threads: 1 on 2 virtual cores Environment: JULIA_IMAGE_THREADS = 1
PackageCompiler v2.1.7
Steps to reproduce
I have reproduced this error on multiple machines. The steps are as follows:
The infinite loop happens inside create_sysimage. This happens with both incremental and non-incremental builds.