panic: runtime error: invalid memory address or nil pointer dereference

[jonaski]

Looks like the tool is crashing here. I'm not familiar with Go code. Looks like it's looking for a "plugins" directory under /usr. This is openSUSE Thumbleweed. The Qt plugins directory is /usr/lib64/qt5/plugins/

jonas@jonas:~/Projects/strawberry/strawberry-appimage/build> ./appimagetool-556-x86_64.AppImage -s deploy AppDir/usr/share/applications/org.strawberrymusicplayer.strawberry.desktop

appimagetool-556-x86_64.AppImage 556

AppDir path: AppDir
Exec= key contains: strawberry
Icon= key contains: strawberry
2020/10/18 01:30:57 Gathering all required libraries for the AppDir...
2020/10/18 01:30:57 Add /usr/lib64/pulseaudio to the libraryLocations directories we search for libraries
2020/10/18 01:30:58 len(allELFsUnderPath): 2
2020/10/18 01:30:58 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio]
2020/10/18 01:30:58 len(allELFs): 138
2020/10/18 01:30:58 Determining Gdk pixbuf loaders (for GDK_PIXBUF_MODULEDIR and GDK_PIXBUF_MODULE_FILE)...
2020/10/18 01:30:58 len(allELFsUnderPath): 15
2020/10/18 01:30:58 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders]
2020/10/18 01:30:58 len(allELFs): 173
2020/10/18 01:30:58 Patching AppDir/usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders.cache removing /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders/
2020/10/18 01:30:58 Bundling GStreamer 1.0 directory (for GST_PLUGIN_PATH)...
2020/10/18 01:30:58 Bundling dependencies of GStreamer 1.0 directory...
2020/10/18 01:30:58 Add /usr/lib64/samba to the libraryLocations directories we search for libraries
2020/10/18 01:30:58 len(allELFsUnderPath): 231
2020/10/18 01:30:58 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders /usr/lib64/gstreamer-1.0 /usr/lib64/samba]
2020/10/18 01:30:58 len(allELFs): 628
2020/10/18 01:30:58 Bundling Gtk 3 directory (for GTK_EXE_PREFIX)...
2020/10/18 01:30:58 Bundling dependencies of Gtk 3 directory...
2020/10/18 01:30:58 len(allELFsUnderPath): 17
2020/10/18 01:30:58 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders /usr/lib64/gstreamer-1.0 /usr/lib64/samba /usr/lib64/gtk-3.0/3.0.0/immodules /usr/lib64/gtk-3.0/3.0.0/printbackends /usr/lib64/gtk-3.0/modules]
2020/10/18 01:30:58 len(allELFs): 655
2020/10/18 01:30:58 Bundling Default theme for Gtk 3 (for GTK_THEME=Default)...
2020/10/18 01:30:58 Bundling alsa-lib directory (for <tbd>)...
2020/10/18 01:30:58 Bundling dependencies of alsa-lib directory...
2020/10/18 01:30:58 len(allELFsUnderPath): 10
2020/10/18 01:30:58 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders /usr/lib64/gstreamer-1.0 /usr/lib64/samba /usr/lib64/gtk-3.0/3.0.0/immodules /usr/lib64/gtk-3.0/3.0.0/printbackends /usr/lib64/gtk-3.0/modules /usr/lib64/alsa-lib]
2020/10/18 01:30:58 len(allELFs): 666
2020/10/18 01:30:58 Bundling pulseaudio directory (for <tbd>)...
2020/10/18 01:30:58 Bundling dependencies of pulseaudio directory...
2020/10/18 01:30:58 len(allELFsUnderPath): 3
2020/10/18 01:30:58 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders /usr/lib64/gstreamer-1.0 /usr/lib64/samba /usr/lib64/gtk-3.0/3.0.0/immodules /usr/lib64/gtk-3.0/3.0.0/printbackends /usr/lib64/gtk-3.0/modules /usr/lib64/alsa-lib]
2020/10/18 01:30:58 len(allELFs): 669
2020/10/18 01:30:58 Deploying /lib64/ld-linux-x86-64.so.2...
2020/10/18 01:30:58 Patching ld-linux...
2020/10/18 01:30:58 Determining gconv (for GCONV_PATH)...
2020/10/18 01:30:59 len(allELFsUnderPath): 253
2020/10/18 01:30:59 libraryLocations: [AppDir/usr/bin /usr/lib64 /lib64 /usr/lib /lib /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /lib32 /usr/lib32 /usr/local/lib64 /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby /usr/lib64/kid3 /usr/X11R6/lib64 /usr/X11R6/lib /usr/lib64/wx-2.8-wxcontainer /usr/lib64/pulseaudio /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders /usr/lib64/gstreamer-1.0 /usr/lib64/samba /usr/lib64/gtk-3.0/3.0.0/immodules /usr/lib64/gtk-3.0/3.0.0/printbackends /usr/lib64/gtk-3.0/modules /usr/lib64/alsa-lib /usr/lib64/gconv]
2020/10/18 01:30:59 len(allELFs): 922
2020/10/18 01:30:59 Adding fontconfig symlink... (is this really the right thing to do?)
2020/10/18 01:30:59 Adding AppRun...
2020/10/18 01:30:59 Find out whether Qt is a dependency of the application to be bundled...
2020/10/18 01:30:59 Detected Qt 5
2020/10/18 01:30:59 Offset of qt_prfxpath: 3778444
2020/10/18 01:30:59 Length of value of qt_prfxpath: 4
2020/10/18 01:30:59 qt_prfxpath: /usr
2020/10/18 01:30:59 Got qt_prfxpath but it does not contain 'plugins'
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x81e0b1]

goroutine 1 [running]:
github.com/probonopd/go-appimage/internal/helpers.FilesWithSuffixInDirectoryRecursive.func1(0xc0003be8a0, 0x28, 0x0, 0x0, 0xa09b60, 0xc0004981b0, 0x0, 0x0)
    github.com/probonopd/go-appimage/internal/helpers/helpers.go:106 +0x41
path/filepath.walk(0xc0001d6220, 0x20, 0xa16240, 0xc0005735f0, 0xc0000eb3c0, 0x0, 0x0)
    path/filepath/path.go:378 +0x20c
path/filepath.walk(0xc00049c700, 0x17, 0xa16240, 0xc000297ad0, 0xc0000eb3c0, 0x0, 0x0)
    path/filepath/path.go:382 +0x2ff
path/filepath.walk(0xc000027f30, 0xe, 0xa16240, 0xc0003145b0, 0xc0000eb3c0, 0x0, 0x0)
    path/filepath/path.go:382 +0x2ff
path/filepath.walk(0xc0004891a0, 0xa, 0xa16240, 0xc0004309c0, 0xc0000eb3c0, 0x0, 0x0)
    path/filepath/path.go:382 +0x2ff
path/filepath.walk(0xc0001f14ac, 0x4, 0xa16240, 0xc0001c9ee0, 0xc0000eb3c0, 0x0, 0xc0000eb2d0)
    path/filepath/path.go:382 +0x2ff
path/filepath.Walk(0xc0001f14ac, 0x4, 0xc0000eb3c0, 0xd25ac0, 0xc0000eb3e0)
    path/filepath/path.go:404 +0xff
github.com/probonopd/go-appimage/internal/helpers.FilesWithSuffixInDirectoryRecursive(0xc0001f14ac, 0x4, 0x96091f, 0xa, 0x8, 0xc0001f14b0, 0xc)
    github.com/probonopd/go-appimage/internal/helpers/helpers.go:105 +0x8e
main.getQtPrfxpath(0xc0001cc428, 0x0, 0x0, 0x5, 0xc0001cc428, 0x0)
    github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:1381 +0x451
main.handleQt(0x7ffc31dd305d, 0x6, 0xc000108980, 0x33, 0xc0000b5ea0, 0x19, 0x5)
    github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:1137 +0x13d
main.AppDirDeploy(0x7ffc31dd305d, 0x4a)
    github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:244 +0xd1d
main.main()
    github.com/probonopd/go-appimage/src/appimagetool/appimagetool.go:133 +0x1145

[probonopd]

Indeed. Looks like the tool failed at identifying the Qt Plugins directory.

2020/10/18 01:30:59 qt_prfxpath: /usr
2020/10/18 01:30:59 Got qt_prfxpath but it does not contain 'plugins'

This means that inside the QtCore library, qt_prfxpath is set to /usr, but /usr/plugins does not exist. So question, why is qt_prfxpath set to /usr in your binary of Qt? Wouldn't it be correct to have qt_prfxpath set to /usr/lib64/qt5/? (That's probably a question only the Qt maintainer for openSUSE can answer...)

[probonopd]

More information: https://github.com/probonopd/linuxdeployqt/issues/79

[probonopd]

Just reading that something has changed recently:

As of version 5.14.0 Qt is relocatable, i.e. it is possible to move the Qt installation to a different directory without breaking functionality or loading of plugins

https://www.qt.io/blog/qt-is-relocatable

Which Qt version are you using? It may well be that the recent changes cannot be handled by the tool correctly yet.

[probonopd]

Details of the recent Qt change: https://codereview.qt-project.org/c/qt/qtbase/+/210936

[jonaski]

It's Qt 5.15.1.

[jonaski]

I managed to make it work by adding: qt_prfxpath = "/usr/lib64/qt5" to the end of the getQtPrfxpath(). I also had to modify the tools variable and remove "uploadtool" from the list. So I think maybe I can use this tool. However I would like to only bundle certain gstreamer plugins, not all of them, and only bundle 1 sql plugin. Maybe we could add environment variables or command line options for setting the Qt path, qt plugins and gstreamer plugins.

[jonaski]

I guess the part in handleGStreamer is where I want to modify it to only include certain plugins, this line: locs, err := findWithPrefixInLibraryLocations("gstreamer-1.0")

[probonopd]

I managed to make it work by adding: qt_prfxpath = "/usr/lib64/qt5" to the end of the getQtPrfxpath().

Is there a way to find out this path programmatically?

I also had to modify the tools variable and remove "uploadtool" from the list.

Why?

However I would like to only bundle certain gstreamer plugins, not all of them, and only bundle 1 sql plugin.

Based on which criteria? Would it be possible to put your criteria into code for an improved autodetection of what to bundle and what not to bundle?

Maybe we could add environment variables or command line options for setting the Qt path, qt plugins and gstreamer plugins.

With this tool, I want to avoid configurability if possible. Everything that can be configured adds complexity, reduces supportability, and will eventually be used incorrectly. Instead. we should be striving for the tool to do the right thing without the need for configuration.

It's always possible to delete files after deploy has been run on the AppDir.

[jonaski]

What I simply suggest is having a environment variable or command line option that can override bundling everything, if it isn't set it can fallback to the behavior you got now bundling everything in gstreamer-1.0. Bundling all the gstreamer plugins will drag in all their dependencies, all sorts of video plugins which it doesn't use, and all the libraries they depend on. Deleting them after bundling isn't an option as it is bugprone and can accidentally delete the wrong dependencies too, and with an gstreamer upgrade might add several more plugins too, which I then have to manually update to dele again. However I realize you are not keen on this idea, so I will just use a modified version of your tool. For now I've simply hardcoded the list in handleGStreamer.

[probonopd]

Bundling all the gstreamer plugins will drag in all their dependencies, all sorts of video plugins which it doesn't use, and all the libraries they depend on. Deleting them after bundling isn't an option as it is bugprone and can accidentally delete the wrong dependencies too, and with an gstreamer upgrade might add several more plugins too, which I then have to manually update to dele again.

I see, you are right about this.

So again my question: How do you decide which gstreamer plugins to bundle and which ones not to bundle?

[jonaski]

I decide it based on what functionality the applications uses in gstreamer, and what relevant audio codecs are needed, etc. ie.: Strawberry got a transcoding feature, so encoding is used too. But gstreamer got a bunch of other audio plugins too that I don't use, and hardly know what is, gstreamer is huge. I don't think it's easy to automatically detect it. Work in progress here: https://github.com/jonaski/go-appimage/tree/strawberry - I don't really know Go lang but at least it runs and looks to be working, have to fiddle a bit with which qt styles I want to include too.