Consider to bundle appstreamcli in the appimagetool AppImage

[kapitainsky]

It works but only when I remove metadata xml file. Otherwise it fails as below:

Found appimagetool: /tmp/.mount_linuxdCqqmwp/usr/bin/appimagetool Running command: /tmp/.mount_linuxdCqqmwp/usr/bin/appimagetool "AppDir" "-g"

appimagetool, continuous build (commit fef038a), build 2093 built on 2019-07-07 12:07:34 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
Using architecture x86_64
/dev/shm/rclone-browser-1.6.0-d9e3ebe.AppImage/AppDir should be packaged as Rclone_Browser-1.6.0-d9e3ebe-x86_64.AppImage
Deleting pre-existing .DirIcon
Creating .DirIcon symlink based on information from desktop file
AppStream upstream metadata found in usr/share/metainfo/rclone-browser.appdata.xml
Trying to validate AppStream information with the appstream-util tool
In case of issues, please refer to https://github.com/hughsie/appstream-glib
/dev/shm/rclone-browser-1.6.0-d9e3ebe.AppImage/AppDir/usr/share/metainfo/rclone-browser.appdata.xml: FAILED:
• url-not-found         : <screenshot> failed to connect: SSL handshake failed [https://github.com/kapitainsky/RcloneBrowser/wiki/images/IMGdefault.png]
• url-not-found         : <screenshot> failed to connect: SSL handshake failed [https://github.com/kapitainsky/RcloneBrowser/wiki/images/IMGtransfer.png]
Validation of files failed
Failed to validate AppStream information with appstream-util
rename: Rclone_Browser*: rename to rclone-browser* failed: No such file or directory
cp: cannot stat ‘*AppImage’: No such file or directory

the same metadata xml file works perfectly on Ubuntu 16.04.

Both urls it complains about are valid.

The issue is in reference to linuxdeploy/linuxdeploy-plugin-appimage#8

[TheAssassin]

gfile.c: In function 'splice_stream_with_progress':
gfile.c:3017:35: error: 'F_SETPIPE_SZ' undeclared (first use in this function)
   buffer_size = fcntl (buffer[1], F_SETPIPE_SZ, 1024 * 1024);

Seems the kernel in CentOS 6 is too old, doubt we can get GLib 2.56 to build there. Unless we identify the section that requires that flag.

[probonopd]

Please make sure your AppImage runs on CentOS 6. Then the app can be included in appimagetool.

It does:

[root@host /]# cat /etc/*-release
CentOS release 6.7 (Final)

[root@host /]# ./appstreamcli-266eccd-x86_64.AppImage --version
(...)
AppStream version: 0.12.10

Then the app can be included in appimagetool.

Well, it's a bit large for that... 23 MB.

[TheAssassin]

You know, building your own stuff on an older distro isn't that hard. I've managed to hack around the issue mentioned above by adding CFLAGS="... -DF_SETPIPE_SZ=1024+7 -DF_GETPIPE_SZ=1024+8. I'm building the remaining containers now. Then I'm going to check we can build appstreamcli in them.

[ximion]

@TheAssassin The absolute minimum version of GLib to build AppStream is now 2.58, and highly likely will remain at that level for quite a while. This is the version that current Debian stable (buster) satisfies, which is a reasonable requirement (and also happens to be a target that I have to support anyway). Ubuntu 20.04 LTS will have at least GLib 2.62. So this should be buildable on any distro unless it's really old.

[benrob0329]

Unsure if I should open a new issue or not, but appstreamcli exits with a symbol lookup error on Fedora 31 when I call appimagetool, but works fine when I call appstreamcli validate directly.

[benrob0329@aleph appimage]$ ~/Applications/appimagetool-x86_64.AppImage AppDir
/usr/bin/AppImageLauncher: /lib64/libcurl.so.4: no version information available (required by /usr/bin/../lib/x86_64-linux-gnu/appimagelauncher/libappimageupdate.so)
appimagetool, continuous build (commit 64321b7), build 2111 built on 2019-11-23 22:20:53 UTC
/home/benrob0329/src/minetest/build/appimage/AppDir/net.minetest.minetest.desktop: warning: key "Keywords" in group "Desktop Entry" is a reserved key for KDE
Using architecture x86_64
/home/benrob0329/src/minetest/build/appimage/AppDir should be packaged as Minetest-x86_64.AppImage
AppStream upstream metadata found in usr/share/metainfo/net.minetest.minetest.appdata.xml
Trying to validate AppStream information with the appstreamcli tool
In case of issues, please refer to https://github.com/ximion/appstream
/usr/bin/appstreamcli: symbol lookup error: /lib64/libappstream.so.4: undefined symbol: g_ptr_array_steal_index_fast
Failed to validate AppStream information with appstreamcli

[benrob0329@aleph appimage]$ appstreamcli validate AppDir/usr/share/metainfo/net.minetest.minetest.appdata.xml 
I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:57
    Consider using a secure (HTTPS) URL for 'http://wiki.minetest.net/FAQ'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:38
    Consider using a secure (HTTPS) URL for 
    'http://www.minetest.net/media/gallery/1.jpg'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:54
    Consider using a secure (HTTPS) URL for 
    'http://www.minetest.net/development/#reporting-issues'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:56
    Consider using a secure (HTTPS) URL for 
    'http://www.minetest.net/development/#donate'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:55
    Consider using a secure (HTTPS) URL for 'http://dev.minetest.net/Translation'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:15
    First 'description/p' paragraph might be too short (< 80 characters).

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:58
    Consider using a secure (HTTPS) URL for 'http://wiki.minetest.net'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:53
    Consider using a secure (HTTPS) URL for 'http://minetest.net'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:41
    Consider using a secure (HTTPS) URL for 
    'http://www.minetest.net/media/gallery/3.jpg'

I - net.minetest.minetest.appdata.xml:net.minetest.minetest.desktop:44
    Consider using a secure (HTTPS) URL for 
    'http://www.minetest.net/media/gallery/5.jpg'

Validation was successful: infos: 10, pedantic: 3

[ximion]

symbol lookup error: /lib64/libappstream.so.4: undefined symbol: g_ptr_array_steal_index_fast Looks like AppStream was built against a newer GLib and used with an older GLib - so the newer version was either not bundled, or AS was compiled against newer GLib by accident.

[probonopd]

@ximion do you know a way to link GLib (or better: everything) statically?

[ximion]

Besides https://github.com/AppImage/AppImageKit/issues/1011#issuecomment-559993203 no. I see no reason why this wouldn't link GLib statically though, provided static libs are available at build time. If that doesn't work, LD_LIBRARY_PATH/LD_PRELOAD with dynamic libraries will definitely do the trick.

[probonopd]

Thanks to the help of @Tachi107 we can now build a static version of appstreamcli, which means that we can use a new appstreamcli version and it will still run on older systems.

This makes bundling appstreamcli in the appimagetool AppImage a real possibility once we have aarch and aarch64 builds, too.

Here you see the static Linux binary running happily on FreeBSD thanks to the Linuxulator:

FreeBSD% file /tmp/appstreamcli-x86_64   
/tmp/appstreamcli-x86_64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped

Man do I love static linking. Well worth the 4,2 MB of disk space (uncompressed).

[ximion]

Neat! :-) But you will not need to build lmdb, AppStream hasn't needed that for a while ;-) Also, on Alpine there apparently might be extremely weird bugs (https://github.com/ximion/appstream/issues/395) possibly related to Musl libc, so it may be useful to at least build the tests (and ideally run them) just to make sure noting major is broken ;-)