Failed process 'gdk-pixbuf-query-loaders.exe' during load on Windows

[Vexatos]

Some of our Windows users are unable to launch our program after the update to Julia 1.3 and Gtk.jl 1.0.0. This appears during launch.

ERROR: LoadError: InitError: failed process: Process(`'C:\Users\snip\.julia\artifacts\382d68de5642eec643713a83a516ccf24e182c79\bin\gdk-pixbuf-query-loaders.exe'`, ProcessExited(3221225781)) [3221225781]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] read(::Cmd) at .\process.jl:412
 [3] (::Gtk.var"#301#307"{String})() at C:\Users\snip\.julia\packages\Gtk\cGgRq\src\Gtk.jl:91
 [4] withenv(::Gtk.var"#301#307"{String}, ::Pair{String,String}) at .\env.jl:161
 [5] (::Gtk.var"#300#306")(::String) at C:\Users\snip\.julia\packages\Gtk\cGgRq\src\Gtk.jl:90
 [6] (::gdk_pixbuf_jll.var"#8#9"{Gtk.var"#300#306"})() at C:\Users\snip\.julia\packages\gdk_pixbuf_jll\76Cuj\src\wrappers\x86_64-w64-mingw32.jl:64
 [7] withenv(::gdk_pixbuf_jll.var"#8#9"{Gtk.var"#300#306"}, ::Pair{String,String}) at .\env.jl:161
 [8] #gdk_pixbuf_query_loaders#7(::Bool, ::Bool, ::typeof(gdk_pixbuf_jll.gdk_pixbuf_query_loaders), ::Gtk.var"#300#306") at C:\Users\snip\.julia\packages\gdk_pixbuf_jll\76Cuj\src\wrappers\x86_64-w64-mingw32.jl:63
 [9] gdk_pixbuf_query_loaders at C:\Users\snip\.julia\packages\gdk_pixbuf_jll\76Cuj\src\wrappers\x86_64-w64-mingw32.jl:47 [inlined]
 [10] __init__() at C:\Users\snip\.julia\packages\Gtk\cGgRq\src\Gtk.jl:89
 [11] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:692
 [12] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:776
 [13] _require(::Base.PkgId) at .\loading.jl:1001
 [14] require(::Base.PkgId) at .\loading.jl:922
 [15] require(::Module, ::Symbol) at .\loading.jl:917
 [16] include at .\boot.jl:328 [inlined]
 [17] include_relative(::Module, ::String) at .\loading.jl:1105
 [18] _require(::Base.PkgId) at .\loading.jl:1053
 [19] require(::Base.PkgId) at .\loading.jl:922
 [20] require(::Module, ::Symbol) at .\loading.jl:917
during initialization of module Gtk
in expression starting at C:\Users\snip\.julia\packages\OurPkg\62IxM\src\OurPkg.jl:10

The line crashing is

using Gtk

Clearly, it's happening during load. Interestingly, it only happens to some Windows users, and none of our developers that use Windows are able to reproduce this issue. Since the executable shipped with gdk_pixbuf_jll appears to be the one erroring, it doesn't seem to be a problem on any other operating system.

[giordano]

This is very timely, yesterday night I faced the same issue for another package on AppVeyor. I don't know what's the problem, my best bet is that there is something weird with the PATH variable.

After installing the gdk_pixbuf_jll package (]add gdk_pixbuf_jll), could you please share the output of

using gdk_pixbuf_jll
gdk_pixbuf_query_loaders() do gdk_pixbuf_query_loaders_path
    ENV[gdk_pixbuf_jll.LIBPATH_env]
end

on both a Windows system where you get the failure and one where it works?

[Vexatos]

This is on a system where it isn't working, I'll try to get it from a working one as quickly as possible. Split into multiple lines for readability, it's a single line in the output.

C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\f7ae2d6c22ec6f7da583da294e1d9d8fec6ff5c5\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\705da39cea730e58ba44dba4e8721a1ed5e8c353\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\753c214aa23d686150c647d64aa5b4f73007a3b1\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\fe89975fcd115e59feb45c086c5c42d8043d349c\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\9a0c1045c9bcae57d0983193f3fa5fe157388e34\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\cf0e3e60225d4ed5192fe910a3ed6e4031295b57\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\217f5fb6408fcad8ec290be23f14646aab4e53b0\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\41065eb36ee97efe783b2b1ca6493e939a41c14a\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\3e07effadfc0e570bee415ace6147fb3a0455938\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\a42da7330f9e38cf60d965fbb89277212c2ee423\\bin;
C:\\Users\\snip\\AppData\\Local\\OurProgram\\juliapkg\\artifacts\\382d68de5642eec643713a83a516ccf24e182c79\\bin;
C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;
C:\\WINDOWS\\system32;
C:\\WINDOWS;
C:\\WINDOWS\\System32\\Wbem;
C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;
C:\\WINDOWS\\System32\\OpenSSH\\;
C:\\Program Files (x86)\\GtkSharp\\2.12\\bin;
C:\\Users\\snip\\AppData\\Local\\Programs\\Python\\Python38-32\\Scripts\\;
C:\\Users\\snip\\AppData\\Local\\Programs\\Python\\Python38-32\\;
C:\\WINDOWS\\System32

We ship our own julia depot as you can see, but it also happens when using properly installed julia, we confirmed that.

[Vexatos]

Checking for the depenencies of the JLL, these artifacts all seem to be properly in the path: cf0e...: Glib_jll 217f...: JpegTurbo_jll 4106...: libpng_jll a42d...: Libtiff_jll

[giordano]

This is on a system where it isn't working, I'll try to get it from a working one as quickly as possible. Split into multiple lines for readability, it's a single line in the output.

Thank you very much. I was hoping that the list was very long, but it's just 1501 characters, it should be short enough.

Still, I'm very interested in seeing the PATH on a system where it works.

We ship our own julia depot as you can see, but it also happens when using properly installed julia, we confirmed that.

Yeah, that shouldn't be a problem, on AppVeyor there is the standard setup.

[Vexatos]

This is one where it worked, in a different depot path.

C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\f7ae2d6c22ec6f7da583da294e1d9d8fec6ff5c5\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\705da39cea730e58ba44dba4e8721a1ed5e8c353\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\753c214aa23d686150c647d64aa5b4f73007a3b1\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\fe89975fcd115e59feb45c086c5c42d8043d349c\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\9a0c1045c9bcae57d0983193f3fa5fe157388e34\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\cf0e3e60225d4ed5192fe910a3ed6e4031295b57\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\217f5fb6408fcad8ec290be23f14646aab4e53b0\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\41065eb36ee97efe783b2b1ca6493e939a41c14a\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\3e07effadfc0e570bee415ace6147fb3a0455938\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\a42da7330f9e38cf60d965fbb89277212c2ee423\\bin
C:\\Users\\snip\\OurProgram\\juliapkg\\artifacts\\382d68de5642eec643713a83a516ccf24e182c79\\bin
C:\\ffmpeg\\bin
C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath
C:\\Program Files\\ImageMagick-7.0.6-Q16
c:\\program files\\graphicsmagick-1.3.26-q8
C:\\ProgramData\\Oracle\\Java\\javapath
C:\\Python27\\Lib\\site-packages\\PyQt4
C:\\Program Files\\ImageMagick-6.8.7-Q16
C:\\Program Files (x86)\\ImageMagick-6.8.6-Q16
C:\\Windows\\system32
C:\\Windows
C:\\Windows\\System32\\Wbem
C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\
C:\\Lua\\5.1
C:\\Lua\\5.1\\clibs
C:\\Python27\\scripts
C:\\Program Files\\7-Zip
C:\\Python27
C:\\Program Files (x86)\\Microsoft SQL Server\\90\\Tools\\binn\\
C:\\strawberry\\c\\bin
C:\\Tools\\
C:\\Program Files (x86)\\GnuWin32\\bin
C:\\Program Files\\Microsoft SQL Server\\110\\Tools\\Binn\\
C:\\Program Files (x86)\\Windows Live\\Shared
C:\\Julia\\Julia-1.1.0\\bin
C:\\Program Files\\FileBot\\
C:\\Program Files (x86)\\GtkSharp\\2.12\\bin
C:\\Ant\\apache-ant-1.10.1\\bin
C:\\WINDOWS\\system32
C:\\WINDOWS
C:\\WINDOWS\\System32\\Wbem
C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\
C:\\Program Files (x86)\\Android\\android-studio\\sdk\\platform-tools
C:\\Program Files\\dotnet\\
C:\\WINDOWS\\System32\\OpenSSH\\
C:\\Program Files\\Microsoft VS Code\\bin
C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\Hostx86\\x86
C:\\Haskell\\ghc-8.8.1\\bin
C:\\Ruby26-x64\\bin
C:\\Users\\snip\\AppData\\Local\\Microsoft\\WindowsApps
C:\\Program Files\\Microsoft VS Code\\bin
C:\\Users\\snip\\AppData\\Roaming\\npm
C:\\Program Files (x86)\\Pandoc
C:\\Users\\snip\\AppData\\Local\\Microsoft\\WindowsApps
C:\\Users\\snip\\AppData\\Local\\Programs\\Microsoft VS Code\\bin

Had to cut (hopefully) irrelevant paths for anonymity, it was 86 entries and just below 4000 characters in total. It's possible that the dependency dll might be in a different directory on this fairly long PATH, allowing Gtk.jl to work...

[giordano]

I was hoping there are unescaped spaces and/or parentheses only in the non-working one, but apparently also the working system has spaces and parentheses in the PATH. I think I'm running out of ideas.

[0x0ade]

Someone with a "working" environment reporting here.

I've just helped someone with a "broken" env and for them, it was caused by libwinpthread-1.dll missing, which is used by libintl-8.dll, used by libgio-2.0-0.dll and libglib-2.0-0.dll.

It cannot be found in either their artifacts dir, nor in mine. On the other hand, I've got libwinpthread-1.dll in another folder in my PATH, which just so happened to not "break" it for me in the first place.

I've shared my copy of the .dll with them and it started working fine.

Hopefully these observations will help you in any way 😅

[Vexatos]

For reference, libintl-8.dll is part of Gettext_jll which is a dependency of Glib_jll. Looks like libwinpthread-1.dll should be part of mingw.

[giordano]

@0x0ade thank you very much, that was really helpful! The library is in Sys.BINDIR, we have to add this directory to the JLL wrappers.

Thanks again for the report and the help!

[giordano]

I have opened a pull request for BinaryBuilder to address this issue: https://github.com/JuliaPackaging/BinaryBuilder.jl/pull/541. Once this is merged, we'll need to rebuild gdk_pixbuf_jll, and then this should finally be fixed.

@Vexatos as a temporary fix you can add the directory where Julia is to the PATH (you can find this directory in Julia with Sys.BINDIR), check that in the same directory there should be the libwinpthread-1.dll library.

[Vexatos]

Thank you! When can we expect a new release of gdk_pixbuf_jll?

[giordano]

As soon as we fix this issue https://github.com/JuliaPackaging/BinaryBuilder.jl/pull/544 that's preventing us from actually building the jll libraries :grimacing: Yesterday there has been a crash while we were building the new gdk_pixbuf_jll.

Did you try the workaround of manually adding Sys.BINDIR to the PATH?

[Vexatos]

I cannot test at the moment, so for now we're just dropping the missing .dll file into the path manually.

[giordano]

A new version of gdk_pixbuf_jll has been released, let us know if this finally solves the issue.

[timholy]

Fixes it on my virtualbox installation!

[Vexatos]

We're getting further now, but there's a new error appearing on the very same machines.

ERROR: LoadError: Couldn’t recognize the image file format for file “C:\Users\snip\AppData\Local\OurProgram\juliapkg\packages\OurPkg\mDNco\docs\logo-256.png”
Stacktrace:
 [1] GError at C:\Users\snip\AppData\Local\OurProgram\juliapkg\packages\Gtk\u6VBX\src\GLib\gerror.jl:17 [inlined]
 [2] #GdkPixbufLeaf#251(::Nothing, ::Nothing, ::String, ::Nothing, ::Nothing, ::Nothing, ::Int64, ::Int64, ::Bool, ::Nothing, ::Type{Gtk.GdkPixbufLeaf}) at C:\Users\snip\.julia\packages\Gtk\u6VBX\src\displays.jl:178
 [3] Type at .\none:0 [inlined]
 [4] #GdkPixbuf#198 at C:\Users\snip\AppData\Local\OurProgram\juliapkg\packages\Gtk\u6VBX\src\GLib\gtype.jl:226 [inlined]
 [5] (::Core.var"#kw#Type")(::NamedTuple{(:filename, :width, :height, :preserve_aspect_ratio),Tuple{String,Int64,Int64,Bool}}, ::Type{Gtk.GdkPixbuf}) at .\none:0
 [6] top-level scope at C:\Users\snip\AppData\Local\OurProgram\juliapkg\packages\OurPkg\mDNco\src\OurPkg.jl:33
in expression starting at C:\Users\snip\AppData\Local\OurProgram\juliapkg\packages\mDNco\src\OurPkg.jl:33

The line in question is just loading our program's logo icon.

const windowIcon = Pixbuf(filename=iconFile, width=-1, height=-1, preserve_aspect_ratio=true)

[giordano]

Is it safe to claim that this new error is unrelated to the original one?

[Vexatos]

it might be. It has occured before on machines that previously had the error with the .exe; in rare cases, the error did not crash julia, and it crashed at this line instead, presumably because gdk-pixbuf-query-loaders.exe failed to recognize .png as a valid file format. Now, the crash at least appears consistently.

[giordano]

Now, the crash at least appears consistently.

Consistency is always important :wink:

Jokes apart, just to clarify: you're now able to load Gtk package and you have to problem only when using it?

[Vexatos]

This is correct, and specifically when loading a Pixbuf.

[giordano]

What is a Pixbuf? Sorry, I'm not familiar with Gtk, I don't have any Pixbuf function exported after loading Gtk

[Vexatos]

https://github.com/JuliaGraphics/Gtk.jl/blob/e243036be95857c5b5c7330fcc0ef91d320d84d3/src/short_exports.jl#L48 You get it from using Gtk.ShortNames.

[giordano]

Ok, Needless to say that your command works for me on a GNU/Linux system, so here starts the fun.

A basic check: make sure that the environment variable ENV["GDK_PIXBUF_MODULE_FILE"] points to an existing file, and the content looks correct, in particular it should reference the PNG format.

[Vexatos]

Indeed, the path points to an existing file, but apart from the autogenerated header, it is completely empty. My own cache properly defines all the file formats.

[giordano]

Indeed, the path points to an existing file, but apart from the autogenerated header, it is completely empty

Ok, good to have found where the problem lies, but this is weird :confused:

@staticfloat, or anyone familiar with Gtk (that is not me :smile:), any clue of what could be the reason why the cache is empty? Did anyone else have this problem? Is something related to this tested in the test suite?

[Vexatos]

According the documentation of gdk-pixbuf-query-loaders, that program is exactly what generates this cache file, meaning it must have failed in some other way now.

https://developer.gnome.org/gdk-pixbuf/stable/gdk-pixbuf-query-loaders.html

[giordano]

Yes, but what could cause it to generate an empty file?

What do you get if you run it manually:

Gtk.gdk_pixbuf_query_loaders() do path
    run(`$path`)
end

? You should see the output on screen

[Vexatos]

All it prints out is the aforementioned header, then it returns the finished Process.

[Vexatos]

Not sure if this is related, but I might as well mention it: Two people that can currently reproduce the issue both have special (cyrillic) characters in their paths, either user or directory names. Could that be a problem?

[giordano]

Honestly now I have no idea what could be the problem, as I'm not familiar with this tool :disappointed: I hope that someone that knows gdk-pixbuf better will chime in.

[staticfloat]

That's a really good guess; can you move your Julia package installation directory to a location that does not have cyrillic characters in the path, install all the packages and try again? You can set this through exporting JULIA_DEPOT_PATH=C:\temp\foo then running Julia.

[Vexatos]

Actually that appears to have worked, at least in one case so far. Fresh install of the depot into a path with no special characters fixed it for the user.

[staticfloat]

So this sounds like an issue in gdk-pixbuf-query-loaders; it must be using an API that doesn't handle non-latin characters properly. If I were to debug this (which I can't, due to time constraints) I would suggest someone look through the code to figure out why special characters are causing issues. Perhaps step through the program with a debugger, etc... Or even just open an issue and see if one of the GNOME devs can help.

[Vexatos]

If anyone has time to find and fix the issue, that would be very much appreciated. The github repo just seems to be a mirror, the real one is here.

[timholy]

@Vexatos, just to be realistic the intersection of (1) Julia developers who are motivated to fix bugs in (2) Gtk that occur only with (3) cyrillic characters might be rather small. In order to see it done, you might need to make that fix yourselves.

[Vexatos]

I am still not certain that this must be a GTK issue. After all, if this were in gdk-pixbuf then this would make gdk-pixbuf not work on virtually any windows install in a language with non-Latin script, and that surely would have been noticed.

[staticfloat]

If you want to double-check, you can try downloading GTK from the official website and installing it to somewhere there are cyrillic characters in the path. Since these installers likely put themselves within C:\Program Files\Gtk or something like that, it's possible that there are no non-latin characters in the default install.

[Vexatos]

Curious that the cache file is generated, just with nothing but the header. Seems to imply that it can properly run and access the cache path, it just doesn't find a single compatible loader...

[Vexatos]

Futher investigation of the code indicates that it is likely related to the path passed as the GDK_PIXBUF_MODULEDIR env variable not being read correctly. That path also contains cyrillic characters, since the entire depot is in such a directory. Could this be an issue with encoding? The env variable is set from within julia and therefore presumably UTF-8 encoded; perhaps gdk-pixbuf or windows expect something else.

[2403772980ygy]

I think this have something to do with windows defender. I have a similar issue(though not on this project), at that time windows defender blocked something. I didn't take a good look at it at that time. Can someone test this?