Precompiling in headless Julia

[Morpheu5]

Hi, I'm trying to set up a headless machine to be used through Jupyter+IJulia. When using ImageView, which in turn uses Gtk, I get the following:

julia> using ImageView
INFO: Precompiling module ImageView.
Failed to connect to Mir: Failed to connect to server socket: No such file or directory
Unable to init server: Could not connect: No such file or directory
ERROR: LoadError: InitError: Cannot open display:
Stacktrace:
 [1] Gtk.GLib.GError(::Gtk.##231#232) at /home/af599/.julia/v0.6/Gtk/src/GLib/gerror.jl:17
 [2] __init__() at /home/af599/.julia/v0.6/Gtk/src/events.jl:10
 [3] _include_from_serialized(::String) at ./loading.jl:157
 [4] _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:200
 [5] _require_search_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:236
 [6] _require(::Symbol) at ./loading.jl:441
 [7] require(::Symbol) at ./loading.jl:405
 [8] include_from_node1(::String) at ./loading.jl:576
 [9] include(::String) at ./sysimg.jl:14
 [10] anonymous at ./<missing>:2
during initialization of module Gtk
while loading /home/af599/.julia/v0.6/ImageView/src/ImageView.jl, in expression starting on line 7
ERROR: Failed to precompile ImageView to /home/af599/.julia/lib/v0.6/ImageView.ji.
Stacktrace:
 [1] compilecache(::String) at ./loading.jl:710
 [2] _require(::Symbol) at ./loading.jl:497
 [3] require(::Symbol) at ./loading.jl:405

julia> using Gtk
Failed to connect to Mir: Failed to connect to server socket: No such file or directory
Unable to init server: Could not connect: No such file or directory
ERROR: InitError: Cannot open display:
Stacktrace:
 [1] Gtk.GLib.GError(::Gtk.##231#232) at /home/af599/.julia/v0.6/Gtk/src/GLib/gerror.jl:17
 [2] __init__() at /home/af599/.julia/v0.6/Gtk/src/events.jl:10
 [3] _include_from_serialized(::String) at ./loading.jl:157
 [4] _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:200
 [5] _require_search_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:236
 [6] _require(::Symbol) at ./loading.jl:441
 [7] require(::Symbol) at ./loading.jl:405
during initialization of module Gtk

I'm on Julia 0.6.1. Is there a way of making this work in a headless setup?

[lobingera]

provide a virtual head? ci on travis includes xvfb (see .travis.yml)

[Morpheu5]

... and that is how one feels like an idiot for not thinking about the most obvious solution :) Sorry for polluting the issue tracker!

[lobingera]

Don't feel bad, focus is really hard to achieve. Feel bad, if you ask the same question twice...

[Morpheu5]

I'm about to do that! I'm getting an endless stream of

(julia:87521): GLib-WARNING **: poll(2) failed due to: Function not implemented.

[lobingera]

(afaik) Gtk(and glib below) not only expect a X11-like display system, but actually a desktop and some connections there (like events from file activities etc). I see a lot of them - when i open my browser f.e. - as i run a windowmanager only and no desktop is available. But the warnings should not be an issue

[Morpheu5]

Well, they start as soon as I run using Gtk and they just hang the REPL. Interestingly enough, that is not a problem inside Jupyter. I may have to think through this whole thing, because I can see how I might want to run Julia scripts that involve ImageView in some ways.

[Morpheu5]

No, the warnings basically hog up stdout/err and Jupyter can't cope with that. I guess that's a problem with Xvfb rather than Gtk.jl, though.

[lobingera]

Could you replace 'Mir'?

[Morpheu5]

I don't believe I have Mir installed aside from libmirclient9 (and deps) which is a dependency of libegl1-mesa and libgtk-3-0.

[lobingera]

Failed to connect to Mir: Failed to connect to server socket: No such file or directory

is contained in your log above

[Morpheu5]

I know, that puzzles me too. I did an apt list --installed | grep -i mir and this is all that comes up

libmirclient9/xenial-updates,xenial-security,now 0.26.3+16.04.20170605-0ubuntu1.1 amd64 [installed,automatic]
libmircommon7/xenial-updates,xenial-security,now 0.26.3+16.04.20170605-0ubuntu1.1 amd64 [installed,automatic]
libmircore1/xenial-updates,xenial-security,now 0.26.3+16.04.20170605-0ubuntu1.1 amd64 [installed,automatic]
libmirprotobuf3/xenial-updates,xenial-security,now 0.26.3+16.04.20170605-0ubuntu1.1 amd64 [installed,automatic]

[NicholasLiang]

Having the same problem and solved.

Install the libgobject-2.0.0.dll for My windows 7 Link to dll4free.com to install libgobject-2.0.0.dll

Move libgobject-2.0.0.dll to C:/Windows/system32/

And then restart Juno/IJulia

Type Pkg.build() to rebuild all the package in METADATA.jl

[AmebaBrain]

For those who connects over ssh to use julia and gets the same error:

$ sudo apt-get update
$ sudo apt-get install -y xvfb xauth
$ xvfb-run julia
julia> using Gtk

These commands are specified in the root .drone.yml file

  commands:
  - apt-get update
  - apt-get install -y xvfb xauth
  - "xvfb-run julia --project=. --check-bounds=yes --color=yes -e 'using InteractiveUtils; versioninfo(verbose=true); using Pkg; Pkg.build(); Pkg.test(coverage=true)'"

[kwshi]

While xvfb-run sounds like a valid workaround to the issue, I want to ask: why does pre-compilation require an X11 connection anyway? Shouldn't it be the case that establishing a X/display connection only happens when the code is run?

[PythonNut]

There are many situations where installing xvfb is not possible, (e.g. HPC). In that case it seems like it is impossible to use Gtk?

[giordano]

There are many situations where installing xvfb is not possible, (e.g. HPC). In that case it seems like it is impossible to use Gtk?

What error message are you seeing?

[thomasjm]

What error message are you seeing?

I'm seeing the same error as on #685:

julia-depot> ERROR: LoadError: InitError: Cannot open display:
julia-depot> Stacktrace:
julia-depot>   [1] error(s::String)
julia-depot>     @ Base ./error.jl:35
julia-depot>   [2] Gtk.GLib.GError(f::Gtk.var"#325#331")
julia-depot>     @ Gtk.GLib /nix/store/2p62vxr126hpxb3llnfyw6dpdn9kgjsf-julia-depot/depot/packages/Gtk/oo3cW/src/GLib/gerror.jl:17
julia-depot>   [3] __init__()
julia-depot>     @ Gtk /nix/store/2p62vxr126hpxb3llnfyw6dpdn9kgjsf-julia-depot/depot/packages/Gtk/oo3cW/src/Gtk.jl:142
julia-depot>   [4] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
julia-depot>     @ Base ./loading.jl:1115
...

As others have said, it's a bit impractical to use xvfb-run just for precompilation in headless environments. Could this library check the DISPLAY environment variable and refrain from trying to initialize GTK when it's not present?

[giordano]

Ok, so that's coming from https://github.com/JuliaGraphics/Gtk.jl/blob/881fdce8eb5a8bce2f1a846f79ac7142177e5b2a/src/Gtk.jl#L142-L146 at least now we know what we're talking about. @tknopp any ideas?

[tknopp]

No, I am also not using Gtk.jl anymore but only Gtk4.jl.

[thomasjm]

@giordano this problem is generally resolved by just adding a check to see if you're in a headless environment before doing things that require a display. It should be fine to bail aggressively in that scenario, because it will only result in less thorough precompilation.

Even better, this StackOverflow answer says there's actually a GTK-specific solution: you call a function called gtk_init_check() in place of the one this package currently uses, and if it returns false, then you refrain from further GTK operations.

[thomasjm]

Is there anything I can do to help move this along? Getting a fix out would be amazing.