Attempting to compile with PackageCompilerX

[IanButterworth]

I have a package that uses CImGui that I'd like to compile down to a sysimage, so I've been trying out PackageCompilerX, but it's having difficulties with CImGui, so I've given compiling CImGui alone a go.

I have the scripts setup to load the demo at examples/demo.jl, which I manually interact with once loaded, to allow PackageCompilerX to generate the precompile statements. https://github.com/ianshmean/CImGui.jl/tree/attempt_compile/dev/compilation

The process succeeds but with various precompile statement execution failures.

[ Info: PackageCompilerX: creating system image object file, this might take a while...
┌ Error: failed to execute precompile(Tuple{Type{Base.Generator{I, F} where F where I}, getfield(Main, Symbol("#17#18")), Base.UnitRange{Int64}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#7#10")), Int64})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#ItemGetter#23")), Ptr{Ptr{Int8}}, Int32, Ptr{Ptr{Int8}}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Saw#25")), Ptr{Nothing}, Int32})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Sin#24")), Ptr{Nothing}, Int32})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Square#14")), Ptr{CImGui.LibCImGui.ImGuiSizeCallbackData}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Step#15")), Ptr{CImGui.LibCImGui.ImGuiSizeCallbackData}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Base.collect), Base.Generator{Base.UnitRange{Int64}, getfield(Main, Symbol("#17#18"))}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowAboutWindow), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindow), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowColumns)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowLayout)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowMisc)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowPopups)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowWidgets)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDummyObject), String, Int64})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppAutoResize), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppConstrainedResize), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppCustomRendering), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppLayout), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppLongText), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppMainMenuBar), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppPropertyEditor), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppSimpleOverlay), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppWindowTitles), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleMenuFile)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowHelpMarker), String})
└ @ Main.anonymous none:23

The sysmimage however does begin to work, but the resulting window has graphical bugs.

i.e.

sh -c "julia-1.3 -J/home/ian/Documents/GitHub/CImGui.jl/dev/compilation/CImGuiSysImage.so -q -e '@show isdefined(Main, :CImGui); include(joinpath(dirname(dirname(pathof(CImGui))), \"examples\", \"demo.jl\"))' -i"

when it should look like

I assume this hasn't been tried before, given how new PackageCompilerX is, but I was wondering if anyone has any ideas? @Gnimuc @kristofferc ?

[IanButterworth]

I guess my key question is whether there's anything inherently limiting about the implementation of CImGui.jl w.r.t. PackageCompilerX's approach?

[KristofferC]

It is not very strange that things like

┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDummyObject), String, Int64})

fail to precompile since those functions only exist in the script and not in the resulting sysimage. In fact, we should probably filter out all precompilation statements that gets defined in Main.

I don't know why the UI doesn't work properly though. Is it possible to get any debug messages out of ClmGui? Perhaps some state is not being set properly. What happens if you run without any precompile stuff? Same thing?

[Gnimuc]

I'm trying to compile this simple app using PackageCompilerX, but it complains about:

julia> create_app("App", "AppCompiled")
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:365
ERROR: Cannot locate artifact 'Xorg_libXau' in '/Users/gnimuc/.julia/packages/Xorg_libXau_jll/VwzXL/Artifacts.toml'
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] #ensure_artifact_installed#41(::Pkg.BinaryPlatforms.MacOS, ::Nothing, ::Bool, ::typeof(Pkg.Artifacts.ensure_artifact_installed), ::String, ::String) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:784
 [3] ensure_artifact_installed(::String, ::String) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:782
 [4] bundle_artifacts(::Pkg.Types.Context, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:538
 [5] #create_app#6(::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(create_app), ::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:443
 [6] create_app(::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:420
 [7] top-level scope at REPL[12]:1

How could I tell it to find the artifacts that used by its dependency packages?

[Gnimuc]

I have the scripts setup to load the demo at examples/demo.jl, which I manually interact with once loaded, to allow PackageCompilerX to generate the precompile statements.

I don't think it's a good idea to compile examples/demo.jl as a proof-of-concept example. This script is directly translated from the C++ code and it just used hundreds of global variables for state management.

[KristofferC]

How could I tell it to find the artifacts that used by its dependency packages?

You shouldn't need to do anything. The error message is a bit strange since the artifact Xorg_libXau should exist in https://github.com/JuliaBinaryWrappers/Xorg_libXau_jll.jl/blob/master/Artifacts.toml...

[KristofferC]

Okay, that package doesn't have any artifact for macOS so we shouldn't try bundle it in that case.

[KristofferC]

Should be fixed by https://github.com/KristofferC/PackageCompilerX.jl/pull/69.

[Gnimuc]

@KristofferC Thanks for the quick fix! Now I hit some @rpath issues:

➜  App git:(compile) ✗ julia -q --project
julia> using PackageCompilerX
[ Info: Precompiling PackageCompilerX [dffaa6cc-da53-48e5-b007-4292dfcc27f1]

shell> cd ..
/Users/gnimuc/.julia/dev/Redux/examples

julia> create_app("App", "AppCompiled")
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:369
[ Info: PackageCompilerX: creating base system image (incremental=false)...
[ Info: PackageCompilerX: creating system image object file, this might take a while...
[ Info: PackageCompilerX: creating system image object file, this might take a while...
ERROR: LoadError: LoadError: error compiling igGET_FLT_MAX: could not load library "@rpath/libcimgui.dylib"
dlopen(@rpath/libcimgui.dylib, 1): image not found
Stacktrace:
 [1] top-level scope at /Users/gnimuc/.julia/packages/CImGui/M9ITG/src/CImGui.jl:22
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] _require(::Base.PkgId) at ./loading.jl:1053
 [5] require(::Base.PkgId) at ./loading.jl:922
 [6] require(::Module, ::Symbol) at ./loading.jl:917
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] _require(::Base.PkgId) at ./loading.jl:1053
 [10] require(::Base.PkgId) at ./loading.jl:922
 [11] require(::Module, ::Symbol) at ./loading.jl:917
in expression starting at /Users/gnimuc/.julia/packages/CImGui/M9ITG/src/CImGui.jl:22
in expression starting at /Users/gnimuc/.julia/dev/Redux/examples/App/src/App.jl:4
caused by [exception 1]
could not load library "@rpath/libcimgui.dylib"
dlopen(@rpath/libcimgui.dylib, 1): image not found
Stacktrace:
 [1] top-level scope at /Users/gnimuc/.julia/packages/CImGui/M9ITG/src/CImGui.jl:22
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] _require(::Base.PkgId) at ./loading.jl:1053
 [5] require(::Base.PkgId) at ./loading.jl:922
 [6] require(::Module, ::Symbol) at ./loading.jl:917
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] _require(::Base.PkgId) at ./loading.jl:1053
 [10] require(::Base.PkgId) at ./loading.jl:922
 [11] require(::Module, ::Symbol) at ./loading.jl:917
ERROR: failed process: Process(`/Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia --color=yes --startup-file=no --compiled-modules=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' --sysimage=/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_E1ghWV/tmp_sys.so --project=/Users/gnimuc/.julia/dev/Redux/examples/App --output-o=/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_eyh60I.o -e 'Base.reinit_stdio()
Base.init_load_path()
Base.init_depot_path()
using App
# This @eval prevents symbols from being put into Main
@eval Module() begin
    PrecompileStagingArea = Module()
    for (_pkgid, _mod) in Base.loaded_modules
        if !(_pkgid.name in ("Main", "Core", "Base"))
            eval(PrecompileStagingArea, :(const $(Symbol(_mod)) = $_mod))
        end
    end
    precompile_statements = String[]
        append!(precompile_statements, readlines("/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_SeD3MB"))

    for statement in sort(precompile_statements)
        # println(statement)
        try
            Base.include_string(PrecompileStagingArea, statement)
        catch
            # See julia issue #28808
            @error "failed to execute $statement"
        end
    end
end # module
    empty!(LOAD_PATH)
    empty!(DEPOT_PATH)
'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] #create_sysimg_object_file#4(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::String, ::Bool, ::typeof(PackageCompilerX.create_sysimg_object_file), ::String, ::Array{Symbol,1}) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:167
 [5] #create_sysimg_object_file at ./none:0 [inlined]
 [6] #create_sysimage#5(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::String, ::String, ::Bool, ::typeof(create_sysimage), ::Symbol) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:275
 [7] (::PackageCompilerX.var"#kw##create_sysimage")(::NamedTuple{(:sysimage_path, :project, :incremental, :precompile_execution_file, :precompile_statements_file, :cpu_target, :base_sysimage, :compiled_modules),Tuple{String,String,Bool,Array{String,1},Array{String,1},String,String,Bool}}, ::typeof(create_sysimage), ::Symbol) at ./none:0
 [8] (::PackageCompilerX.var"#7#8"{Array{String,1},Array{String,1},Bool,Bool,String,String})() at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:463
 [9] cd(::PackageCompilerX.var"#7#8"{Array{String,1},Array{String,1},Bool,Bool,String,String}, ::String) at ./file.jl:104
 [10] #create_app#6(::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(create_app), ::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:452
 [11] create_app(::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:424
 [12] top-level scope at REPL[3]:1

I guess this is not supposed to happen since CImGui.jl has already switched to the new artifacts build system and hence there shouldn't be any re-locatability issues described here.

[KristofferC]

Fixing https://github.com/KristofferC/PackageCompilerX.jl/issues/68 might help this. I will try it out today.

[KristofferC]

Ok with https://github.com/KristofferC/PackageCompilerX.jl/pull/71 creating the app works well.

However, it immediately exits when started. Perhaps nothing is waiting for the UI task?

[KristofferC]

Actually, it is a bug in PackageCompilerX.

[IanButterworth]

This is all fantastic. Thanks!! I’m looking forward to testing it out when back at a computer tomorrow. I can see if that last bug occurs for my package.

[KristofferC]

Got it working with incremental=true at least:

[IanButterworth]

I gave this a go on my system but the app window still immediately closes when runing bin/App. I'm on the up to date kc/compilecache branch, with 1.3.1 on ubuntu 18

[Gnimuc]

@KristofferC I hit https://github.com/Gnimuc/CImGui.jl/issues/15#issuecomment-570838822 again after you force-pushed KristofferC/PackageCompilerX.jl#71.

[KristofferC]

@Gnimuc

The log showing:

Base.init_load_path()
Base.init_depot_path()
using App

maks me think that it is not running the version in the PR. It should never run using anymore:

https://github.com/KristofferC/PackageCompilerX.jl/pull/71/files#diff-c8c207e0820e5dba4d133f965fa4c6ceL114-R133

[Gnimuc]

indeed, I forgot to change the branch 🤦‍♂ . now I get:

julia> create_app("App", "AppCompiled", incremental=true, force=true)
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:431
[ Info: PackageCompilerX: creating system image object file, this might take a while...
fatal: error thrown and no exception handler available.
ErrorException("error compiling julia_main: error compiling init_renderer: could not load library "@rpath/libglfw.3.dylib"
dlopen(@rpath/libglfw.3.dylib, 1): image not found")
rec_backtrace at /Users/sabae/buildbot/worker/package_macos64/build/src/stackwalk.c:94
record_backtrace at /Users/sabae/buildbot/worker/package_macos64/build/src/task.c:224
jl_throw at /Users/sabae/buildbot/worker/package_macos64/build/src/task.c:461
jl_rethrow_with_add at /Users/sabae/buildbot/worker/package_macos64/build/src/codegen.cpp:791
jl_compile_linfo at /Users/sabae/buildbot/worker/package_macos64/build/src/codegen.cpp:1205
_generate_from_hint at /Users/sabae/buildbot/worker/package_macos64/build/src/gf.c:2007 [inlined]
jl_compile_now at /Users/sabae/buildbot/worker/package_macos64/build/src/gf.c:2015
jl_precompile at /Users/sabae/buildbot/worker/package_macos64/build/src/precompile.c:388
jl_write_compiler_output at /Users/sabae/buildbot/worker/package_macos64/build/src/precompile.c:34
jl_atexit_hook at /Users/sabae/buildbot/worker/package_macos64/build/src/init.c:230
main at /Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia (unknown line)
ERROR: failed process: Process(`/Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia --color=yes --startup-file=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' --sysimage=/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia/sys.dylib --project=/Users/gnimuc/.julia/dev/Redux/examples/App --output-o=/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_cGlj1j.o -e 'Base.reinit_stdio()
Base.init_load_path()
Base.init_depot_path()
@eval Module() begin
    m = Base._require_from_serialized("/Users/gnimuc/.julia/compiled/v1.3/App/6XOLX_GDaLN.ji")
    m isa Exception && throw(m)
end
# This @eval prevents symbols from being put into Main
@eval Module() begin
    PrecompileStagingArea = Module()
    for (_pkgid, _mod) in Base.loaded_modules
        if !(_pkgid.name in ("Main", "Core", "Base"))
            eval(PrecompileStagingArea, :(const $(Symbol(_mod)) = $_mod))
        end
    end
    precompile_statements = String[]
        append!(precompile_statements, readlines("/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_vSd7O3"))

    for statement in sort(precompile_statements)
        # println(statement)
        try
            Base.include_string(PrecompileStagingArea, statement)
        catch
            # See julia issue #28808
            @error "failed to execute $statement"
        end
    end
end # module
Base.@ccallable function julia_main()::Cint
    try
        app = Base.loaded_modules[Base.PkgId(Base.UUID("f943f3d7-887a-4ed5-b0c0-a1d6899aa8f5"), "App")]
        app.julia_main() 
    catch
        return 1
    end
end
empty!(LOAD_PATH)
empty!(DEPOT_PATH)
'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] #create_sysimg_object_file#4(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::String, ::Base.PkgId, ::typeof(PackageCompilerX.create_sysimg_object_file), ::String, ::Array{String,1}) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:200
 [5] #create_sysimg_object_file at ./none:0 [inlined]
 [6] #create_sysimage#5(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::String, ::Nothing, ::Base.PkgId, ::typeof(create_sysimage), ::Symbol) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:334
 [7] (::PackageCompilerX.var"#kw##create_sysimage")(::NamedTuple{(:sysimage_path, :project, :incremental, :filter_stdlibs, :precompile_execution_file, :precompile_statements_file, :cpu_target, :app),Tuple{String,String,Bool,Bool,Array{String,1},Array{String,1},String,Base.PkgId}}, ::typeof(create_sysimage), ::Symbol) at ./none:0
 [8] (::PackageCompilerX.var"#9#10"{Array{String,1},Array{String,1},Bool,Bool,String,Base.PkgId,String})() at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:534
 [9] cd(::PackageCompilerX.var"#9#10"{Array{String,1},Array{String,1},Bool,Bool,String,Base.PkgId,String}, ::String) at ./file.jl:104
 [10] #create_app#8(::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(create_app), ::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:515
 [11] (::PackageCompilerX.var"#kw##create_app")(::NamedTuple{(:incremental, :force),Tuple{Bool,Bool}}, ::typeof(create_app), ::String, ::String) at ./none:0
 [12] top-level scope at REPL[3]:1

[KristofferC]

Are you on 1.3.1? That is required for PackageCompilerX.

[Gnimuc]

no. 🤦‍♂ BB2 has some problems with 1.3.1 so I'm still using 1.3.0.

[KristofferC]

Okay, there are some bugfixes in 1.3.1 that are needed, unfortunately (in this case https://github.com/JuliaLang/julia/pull/34062)

[Gnimuc]

With the latest https://github.com/KristofferC/PackageCompilerX.jl/pull/71, I can compile it successfully, but it immediately exits even with incremental=true:

shell> cd ~/.julia/dev/Redux/examples/App
/Users/gnimuc/.julia/dev/Redux/examples/App

julia> using PackageCompilerX
[ Info: Precompiling PackageCompilerX [dffaa6cc-da53-48e5-b007-4292dfcc27f1]

(v1.3) pkg> activate .
Activating environment at `~/.julia/dev/Redux/examples/App/Project.toml`

(App) pkg> st
Project App v0.1.0
    Status `~/.julia/dev/Redux/examples/App/Project.toml`
  [5d785b6c] CImGui v1.74.1
  [6d653af8] Redux v0.1.0 [`~/.julia/dev/Redux`]

julia> using PackageCompilerX

julia> create_app("../App", "../AppCompiled", incremental=true)
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:447
[ Info: PackageCompilerX: creating system image object file, this might take a while...

julia> run(`../AppCompiled/bin/App`)
Process(`../AppCompiled/bin/App`, ProcessExited(0))

[Gnimuc]

OK, it's working(both incremental=true and incremental=false) if I remove the @sync call from this line, but there're wired graphic bugs too:

[KristofferC]

OK, it's working if I remove the @sync call from this line

Yeah, something has to wait on the render task or everything just falls through and the main function returns.

but there're wired graphic bugs too:

I cannot repro... Strange (I'm on Linux now, my prev screenshot was mac)

[KristofferC]

I got the graphics bug too now...

[Gnimuc]

😂 Only on macOS? I moved some source code out of the App folder on Redux.jl's master. Could that be the reason for the bug?

[KristofferC]

No on Linux as well. Trying to figure out why... I didn't update Redux so shouldn't be the reason...

[KristofferC]

I only get the graphics bug when I run with a precompilation file...:

[IanButterworth]

Could it be linked to the speed of execution being faster and GLFW not being ready? I mean it shouldn’t, but the speed of execution is maybe the only difference?

[Gnimuc]

I don't know what job a precompilation file does in this case(I also didn't set up one when I was doing tests). Does that imply I did something wrong when setting up those global variables in __init__ here and here?

[KristofferC]

The precompile file should in theory just speed up the execution by removing some need to compile functions. And since you get the graphics bug even without it, it feels like precompilation or not isn't the end of the story.

Maybe the DearImGui guys have some idea what would cause problems like this...

[KristofferC]

For posterity, https://github.com/Gnimuc/CImGui.jl/pull/16 didn't change anything when it comes to the graphics bugs.

[Gnimuc]

Confirmed this is a CImGui issue. It turns out the naive OpenGL/GLFW example in this branch works well with PackageCompilerX.

[Gnimuc]

looks like it's working now...

versioninfo:

• Julia 1.3.1
• PackageCompilerX#master
• Redux#master
• CImGui#ogl-backend-update

[IanButterworth]

I'm afraid it's still immediately closing for me:

(v1.3) pkg> st
    Status `~/.julia/environments/v1.3/Project.toml`
  [5d785b6c] CImGui v1.74.1 #ogl-backend-update (https://github.com/Gnimuc/CImGui.jl.git)
  [ea656a56] CSyntax v0.3.0
  [66fc600b] ModernGL v1.1.2
  [dffaa6cc] PackageCompilerX v0.1.0 [`~/Documents/GitHub/PackageCompilerX.jl`]
  [6d653af8] Redux v0.1.0 [`~/.julia/dev/Redux`]

(v1.3) pkg> free PackageCompilerX
ERROR: cannot free a `dev`ed package that does not exist in a registry

julia> create_app("App", "AppCompiled", force=true, incremental=true)
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/Documents/GitHub/PackageCompilerX.jl/src/PackageCompilerX.jl:432
precompile_paths = ["/home/ian/.julia/compiled/v1.3/App/6XOLX_l6U19.ji"]
[ Info: PackageCompilerX: creating system image object file, this might take a while...

julia> run(`AppCompiled/bin/App`)
Process(`AppCompiled/bin/App`, ProcessExited(0))

[KristofferC]

With https://github.com/Gnimuc/Redux.jl/pull/5/files#diff-3273ac476f7170cd30f851d83be23287R28 it shouldn't. Did you update the Redux branch to that?

[IanButterworth]

Ah, I was on an outdated fork master. Now with that PR, the App stays open but:

[IanButterworth]

Also, when I manually close the window I get:

julia> run(`AppCompiled/bin/App`)
ERROR: MethodError: no method matching wait(::Nothing)
Closest candidates are:
  wait(::Base.Event) at lock.jl:286
  wait(::Task) at task.jl:248
  wait(::Base.Process) at process.jl:615
  ...
Stacktrace:
 [1] real_main() at /home/ian/Documents/GitHub/Redux.jl/examples/App/src/App.jl:28
 [2] julia_main() at /home/ian/Documents/GitHub/Redux.jl/examples/App/src/App.jl:17
 [3] julia_main() at ./none:32
ERROR: failed process: Process(`AppCompiled/bin/App`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
 [3] run(::Cmd) at ./process.jl:438
 [4] top-level scope at none:0

[IanButterworth]

t = Renderer.render(()->todo_mvc_ui(store), width=600, height=300, title="App: TodoMVC")
wait(t)

doesn't seem right, and when I revert those lines it stays open and closes correctly.. I've lost track of what change actually fixed the window not staying open..

[Gnimuc]

@KristofferC @ianshmean the graphic bug should be fixed now, see https://github.com/Gnimuc/Redux.jl/tree/master/examples/App.

[IanButterworth]

Fixed for me on Ubuntu. This is great! Thanks so much

[KristofferC]

This isn't related to compilation but the Redux App example looks like:

for me on Mac.

So it seems like some size is being calculated incorrectly since the buttons in the bottom are not visisble, for example.

[Gnimuc]

That should be fixed by this commit, please try the latest Redux.jl's master again.

BTW, I also released a precompiled-binary for macOS here.

[KristofferC]

Hm, it doesn't seem fixed to me. I'm using the latest master.

And boy, is Catalina annoying. Every single library needs to be explicitly approved:

[Gnimuc]

I can not reproduce... Do you get the same result when running https://github.com/Gnimuc/Redux.jl/blob/master/examples/todomvc/app.jl directly?

[KristofferC]

I tried this again now (new computer) and I am still getting the oversized window (even when using app.jl directly:

[Gnimuc]

@KristofferC I still cannot reproduce, could you give this https://github.com/Gnimuc/Redux.jl/pull/8 a try?

[KristofferC]

If I change the window size for CimGui in the render loop to be half of the display sizes:

CImGui.SetNextWindowSize((div(display_w, 2), div(display_h, 2)))

in the render loop, it works perfectly:

My guess is that it has something to do with DPI and scale factors, e.g as described in https://docs.rs/winit/0.21.0/winit/dpi/index.html#how-is-the-scale-factor-calculated:

macOS: "retina displays" have a scale factor of 2.0. Otherwise, the scale factor is 1.0. Intermediate scale factors are never used. It's possible for any display to use that 2.0 scale factor, given the use of the command line.

That is why I tried to divide by two since I thought maybe there is some disagreement somewhere if things are measured in pixels with or without scale factor taken into account.

[Gnimuc]

Thanks for the info! I now can reproduce the issue on a retina display. I've pushed a new fix on https://github.com/Gnimuc/Redux.jl/pull/8.

[KristofferC]

Yep, works perfect with the PR! Thanks :)