Beep6581 / RawTherapee

A powerful cross-platform raw photo processing program
https://rawtherapee.com
GNU General Public License v3.0
2.84k stars 321 forks source link

Automated AppImage and Windows builds with Travis CI #4404

Closed aferrero2707 closed 4 years ago

aferrero2707 commented 6 years ago

Dear all, I have just finished setting up a GitHub repository to automate the building and packaging of RT for Linux (in AppImage format) and Windows 64-bit (cross-compiled with mingw-w64). The build process is based on Travis CI, and uses custom Docker containers that provide the required dependencies. The packages are automatically uploaded back to GitHub.

At the moment the building is limited to the devRT branch, but other branches (like releases and newlocallab) can be easily added. The build process can be triggered by hand or run regularly (daily or weekly) using cron jobs.

It should also be possible to include OSX packages in the automated building process.

The advantage of this approach is that each build is executed in a well-defined and encapsulated environment, and is perfectly reproducible. Moreover, any user that is added as a collaborator to the github repository should also be able to trigger new builds on Travis, without the need of having a windows system at hand...

First of all, and before doing further developments of the build environment, I'd like to know if the RT developers are interested in this possibility. Of course I'm open to any suggestion and remark!

Beep6581 commented 6 years ago

Hello @aferrero2707

  1. The AppImage was updated correctly.
  2. I totally missed that the Zenity dialog showed the download destination - I looked only in the console output and could not find where the AppImage was saved. Would be nice if the console output made that clear as well.
  3. The console output mentions some download_release.sh script which does not exist.
  4. The download progress gets lost in the sea of other output. Separating it from the rest with a blank line before and after would be good.

Patch for the above:

diff --git a/AppRun b/AppRun
index 1828ed7..757f7d8 100755
--- a/AppRun
+++ b/AppRun
@@ -39,10 +39,15 @@ if [ "x" = "x" ]; then
        "$APPDIR/usr/bin/zenity.sh"  --question --text "New version available:\n\nhttps://github.com/${REPO_SLUG}/releases/${RELEASE_TAG}/${AI}\n\nDo you want to download and run it?\nThe new package will be stored in $HOME/.local/appimages"
        ANSWER=$?
        if [ x"$ANSWER" = "x0" ]; then
-           echo "./download_release.sh \"$REPO_SLUG\" \"$AI\" \"$ID\""
-           mkdir -p "$HOME/.local/appimages"
+           mkdir -p "$HOME/.local/appimages" | exit 1
            asset_url="https://api.github.com/repos/$REPO_SLUG/releases/assets/$ID"
+           printf '%s\n' "" \
+               "Downloading latest RawTherapee AppImage" \
+               "    From: ${asset_url}" \
+               "      To: $HOME/.local/appimages/$AI" \
+               "Progress:"
            curl -XGET -o "$HOME/.local/appimages/$AI" --location --header "Accept: application/octet-stream" "${asset_url}"
+           printf '\n'
            chmod u+x "$HOME/.local/appimages/$AI"
            "$HOME/.local/appimages/$AI"
            exit
aferrero2707 commented 6 years ago

@Beep6581 Thanks for checking! Do you find this worth to be included in next appimage releases?

Also, did you have a chance to check the desktop integration? From a user perspective, this is also a very important aspect...

Beep6581 commented 6 years ago

Do you find this worth to be included in next appimage releases?

Yes please.

Also, did you have a chance to check the desktop integration?

Ah yes, I forgot to mention that this part didn't appear, I did not see any "Would you like to integrate" window.

aferrero2707 commented 6 years ago

@Beep6581

I did not see any "Would you like to integrate" window.

Do you have the desktop-file-install command in your system? Under CentOS, it is available through the desktop-file-utils package, and in Sabayon it seems to be called in the same way.

This command is mandatory in order to install a new *.desktop file and add the proper entries in the applications menu.

I will add a warning in the console messages if that is missing.

Beep6581 commented 6 years ago

Yes, I have desktop-file-edit, desktop-file-install and desktop-file-validate, they're from dev-util/desktop-file-utils-0.23.

aferrero2707 commented 6 years ago

@Beep6581

Could you check the contents of your $HOME/.local/share/appimagekit/ folder and make sure there is no file called RawTherapee_no_desktopintegration?

If it is present, please delete it and try running again the AppImage...

Beep6581 commented 6 years ago

@aferrero2707 there was an empty file ~/.local/share/appimagekit/RawTherapee_no_desktopintegration Where did it come from? Sorry if this is obvious by looking at the code, but I won't be able to check for myself until tomorrow.

aferrero2707 commented 6 years ago

@Beep6581

This was probably created in the past when you answered "Yes" to this sort of dialog: screen shot 2018-09-27 at 15 19 37

If you remove it, you should be prompted with a request to integrate RawTherapee in your desktop...

aferrero2707 commented 6 years ago

@Beep6581 I have integrated your patch, and prepared new packages. Apart from the automated updating, the other important improvement concerns the build environment. The new packages are built using a base CantOS7 docker image, where all the dependencies (binary and compiled from sources) are installed when the AppImage gets created. This insures that all packages are up-to-date and include the latest security patches.

I consider the automated updating still as an experimental feature, so I disabled it by default - at least until the feature has been tested a bit more. To enable the automated updating, one has to pass the --update option when invoking the AppImage.

A question to the developers (@Beep6581 @heckflosse @SilvioGrosso and others): what is the preferred GCC version for compiling RT?

The new packages are available as usual from the "nightly" builds release.

heckflosse commented 6 years ago

@aferrero2707

what is the preferred GCC version for compiling RT?

I don't think there is a preferred GCC version for building RT. It should build fine from 4.9.x on upwards. Though, as your question implies a choice I would say, the higher the better, but it does not have to be bleeding edge

Beep6581 commented 6 years ago

@aferrero2707 if I use automated updating, can I be sure that the packages will remain isolated in my system? Where will they be downloaded and built to?

aferrero2707 commented 6 years ago

@Beep6581

if I use automated updating, can I be sure that the packages will remain isolated in my system? Where will they be downloaded and built to?

Sorry, I think there is some confusion on this point. When the AppImage is updated, there are no dependencies that are downloaded and built. Only the new AppImage is downloaded, which is a self-contained package that does not "pollute" your system. The dependencies are updated and built on TravisCI, when the AppImage is generated.

I hope this clarifies your doubts. Otherwise, you can always inspect the contents of the AppImage, by passing the --appimage-extract option. All the startup procedure is based on plain-text bash scripts, and the entry point is AppRun. The same scripts can be inspected also on GitHub. For example, the download of the updated AppImage package is done here. As you can see, nowhere you have instructions to install or build dependencies on the host system...

aferrero2707 commented 6 years ago

The next thing I would like to integrate into the RT AppImage, particularly in connection with automated updates, is package signature and validation.

The AppImage format allows to embed a GPG signature, so that the integrity of the packages can be verified after download.

This is how I propose to organise the digital signing of the RT AppImage packages, using GPG:

  1. a trusted RT developer generates a private/public GPG key pair with gpg2 --full-gen-key, starting with an empty $HOME/.gpg folder
  2. the same user creates a tar archive of the .gpg folder containing the new private/public pair:
    cd $HOME
    tar czf keys.tgz .gpg
  3. the same user encrypts the tar archive, taking note of the used passphrase
  4. the encrypted tar archive is uploaded to the “nightly builds” release
  5. the same RT user adds the encryption passprhase used at step 3. as an hidden environment variable on the TravisCI repository used for building the AppImages (provided this is possible for somebody who is not the owner of the repository, otherwise the passphrase will have to be communicated to me in some way). The decryption key itself will be kept encrypted on Travis, and will not be exposed in any of the logs. More info here
  6. During the build jobs on Travis, and before generating the AppImage package, the tar archive with the GPG keys will be downloaded, de-crypted and extracted into the $HOME directory of the job.
  7. The GPG keys will then be used to sign the AppImage package when it is generated
  8. The public GPG key is uploaded together with the AppImage package to the “nightly builds” release
  9. During the update process, the downloaded RT AppImages will be verified against the GPG public key, and discarded if they do not match.

An example can be found here.

What do you think?

aferrero2707 commented 6 years ago

@Beep6581 I have uploaded new AppImage packages in the nightly builds release, and added the auto-update feature. It would be great if you could double-check that everything works as expected... The auto-update is disabled by default - it must be explicitly enabled with

./RawTherapee-dev-5.4-918-gae12a41.AppImage --enable-auto-update

I wrote some more detailed informations and instructions here

Thanks!

Beep6581 commented 6 years ago

Hey @aferrero2707

I downloaded RawTherapee-dev-5.4-922-g404ff5e.AppImage from https://github.com/Beep6581/RawTherapee/releases/tag/nightly

I corrupted it and ran it. It ran successfully. The AppImage did not detect that it was corrupted:

morgan@sabayon ~/downloads $ cp RawTherapee-dev-5.4-922-g404ff5e.AppImage RawTherapee-dev-5.4-922-g404ff5e_corrupt.AppImage

morgan@sabayon ~/downloads $ dd if=/dev/zero of=RawTherapee-dev-5.4-922-g404ff5e_corrupt.AppImage bs=1 count=1 seek=40000000 conv=notrunc
1+0 records in
1+0 records out
1 byte copied, 0.000460757 s, 2.2 kB/s

morgan@sabayon ~/downloads $ sha256sum RawTherapee-dev-5.4-922-g404ff5e*
17f687b630994096f3a70513131ea7426d8cfaedf9dc253d94a1b8c2d1a51b47  RawTherapee-dev-5.4-922-g404ff5e.AppImage
cf8f42c0aead8c5605aafe68491d0bcc116b43bf754eacfd20a6e2c3228fb2a8  RawTherapee-dev-5.4-922-g404ff5e_corrupt.AppImage

morgan@sabayon ~/downloads $ rm -rf ~/.config/RawTherapee5-dev-ai ~/.cache/RawTherapee5-dev-ai 

morgan@sabayon ~/downloads $ ./RawTherapee-dev-5.4-922-g404ff5e_corrupt.AppImage 
AILIBDIR=/tmp/tmp.lVQZh5gl96
deleting /tmp/tmp.lVQZh5gl96/libxcb-dri3.so*
System stdc++ library: "/usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libstdc++.so.6"
System stdc++ library version: "3.4.22"
Bundled stdc++ library version: "3.4.19"
Newest stdc++ library version: "3.4.22"
Using system stdc++ library
atomiclib: /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libatomic.so.1
Checking versions of library "libfontconfig"
  system library: "/usr/lib64/libfontconfig.so.1.11.1" (libfontconfig.so.1.11.1)
  system library version: 1.11.1
  ailib: "/tmp/.mount_RawThek3vKcP/usr/lib/libfontconfig.so.1.7.0"
  bundled library: "/tmp/.mount_RawThek3vKcP/usr/lib/libfontconfig.so.1.7.0" (libfontconfig.so.1.7.0)
  bundled library version: 1.7.0
  newest library version: "1.11.1"
Removing bundled "libfontconfig"
removed '/tmp/tmp.lVQZh5gl96/libfontconfig.so.1'
removed '/tmp/tmp.lVQZh5gl96/libfontconfig.so.1.7.0'
Checking versions of library "libfreetype"
  system library: "/usr/lib64/libfreetype.so.6.16.1" (libfreetype.so.6.16.1)
  system library version: 6.16.1
  ailib: "/tmp/.mount_RawThek3vKcP/usr/lib/libfreetype.so.6.10.0"
  bundled library: "/tmp/.mount_RawThek3vKcP/usr/lib/libfreetype.so.6.10.0" (libfreetype.so.6.10.0)
  bundled library version: 6.10.0
  newest library version: "6.16.1"
Removing bundled "libfreetype"
removed '/tmp/tmp.lVQZh5gl96/libfreetype.so.6'
removed '/tmp/tmp.lVQZh5gl96/libfreetype.so.6.10.0'
DO_UPDATE: 1
GTK_PATH=/tmp/.mount_RawThek3vKcP/usr/lib/gtk-2.0
GTK_IM_MODULE_FILE=/tmp/.mount_RawThek3vKcP/usr/lib/gtk-2.0:/tmp/.mount_RawThek3vKcP/usr/lib/gtk-2.0
PANGO_LIBDIR=/tmp/.mount_RawThek3vKcP/usr/lib
        linux-vdso.so.1 (0x00007ffd0e3ef000)
        libcanberra-gtk3.so.0 => /tmp/tmp.lVQZh5gl96/libcanberra-gtk3.so.0 (0x00007f570d994000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0x00007f570d63d000)
        libcanberra.so.0 => /tmp/tmp.lVQZh5gl96/libcanberra.so.0 (0x00007f570d42c000)
        libgtk-3.so.0 => /tmp/tmp.lVQZh5gl96/libgtk-3.so.0 (0x00007f570cb19000)
        libgdk-3.so.0 => /tmp/tmp.lVQZh5gl96/libgdk-3.so.0 (0x00007f570c82d000)
        libatk-1.0.so.0 => /tmp/tmp.lVQZh5gl96/libatk-1.0.so.0 (0x00007f570c607000)
        libgio-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libgio-2.0.so.0 (0x00007f570c26c000)
        libpangocairo-1.0.so.0 => /tmp/tmp.lVQZh5gl96/libpangocairo-1.0.so.0 (0x00007f570c05f000)
        libgdk_pixbuf-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libgdk_pixbuf-2.0.so.0 (0x00007f570be37000)
        libcairo-gobject.so.2 => /tmp/tmp.lVQZh5gl96/libcairo-gobject.so.2 (0x00007f570bc2e000)
        libpango-1.0.so.0 => /tmp/tmp.lVQZh5gl96/libpango-1.0.so.0 (0x00007f570b9e3000)
        libcairo.so.2 => /tmp/tmp.lVQZh5gl96/libcairo.so.2 (0x00007f570b6bb000)
        libgobject-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libgobject-2.0.so.0 (0x00007f570b46a000)
        libglib-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libglib-2.0.so.0 (0x00007f570b156000)
        libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007f570af27000)
        libatomic.so.1 => /tmp/tmp.lVQZh5gl96/libatomic.so.1 (0x00007f570ad1f000)
        libfftw3f.so.3 => /tmp/tmp.lVQZh5gl96/libfftw3f.so.3 (0x00007f570a9a6000)
        libfftw3f_omp.so.3 => /tmp/tmp.lVQZh5gl96/libfftw3f_omp.so.3 (0x00007f570a79f000)
        libgiomm-2.4.so.1 => /tmp/tmp.lVQZh5gl96/libgiomm-2.4.so.1 (0x00007f570a3e1000)
        libglibmm-2.4.so.1 => /tmp/tmp.lVQZh5gl96/libglibmm-2.4.so.1 (0x00007f570a161000)
        libsigc-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libsigc-2.0.so.0 (0x00007f5709f5a000)
        libgthread-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libgthread-2.0.so.0 (0x00007f5709d58000)
        libgtkmm-3.0.so.1 => /tmp/tmp.lVQZh5gl96/libgtkmm-3.0.so.1 (0x00007f57095aa000)
        libatkmm-1.6.so.1 => /tmp/tmp.lVQZh5gl96/libatkmm-1.6.so.1 (0x00007f570935c000)
        libgdkmm-3.0.so.1 => /tmp/tmp.lVQZh5gl96/libgdkmm-3.0.so.1 (0x00007f5709108000)
        libpangomm-1.4.so.1 => /tmp/tmp.lVQZh5gl96/libpangomm-1.4.so.1 (0x00007f5708eda000)
        libcairomm-1.0.so.1 => /tmp/tmp.lVQZh5gl96/libcairomm-1.0.so.1 (0x00007f5708cb4000)
        libiptcdata.so.0 => /tmp/tmp.lVQZh5gl96/libiptcdata.so.0 (0x00007f5708aa8000)
        libjpeg.so.62 => /tmp/tmp.lVQZh5gl96/libjpeg.so.62 (0x00007f5708853000)
        liblcms2.so.2 => /tmp/tmp.lVQZh5gl96/liblcms2.so.2 (0x00007f57085f9000)
        libpng15.so.15 => /tmp/tmp.lVQZh5gl96/libpng15.so.15 (0x00007f57083ce000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f57081b5000)
        libtiff.so.5 => /tmp/tmp.lVQZh5gl96/libtiff.so.5 (0x00007f5707f3e000)
        liblensfun.so.1 => /tmp/tmp.lVQZh5gl96/liblensfun.so.1 (0x00007f5707d21000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f57079cd000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libstdc++.so.6 (0x00007f570759d000)
        libgomp.so.1 => /tmp/tmp.lVQZh5gl96/libgomp.so.1 (0x00007f5707377000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libgcc_s.so.1 (0x00007f5707160000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5706f3f000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f5706b6a000)
        libvorbisfile.so.3 => /tmp/tmp.lVQZh5gl96/libvorbisfile.so.3 (0x00007f5706961000)
        libvorbis.so.0 => /tmp/tmp.lVQZh5gl96/libvorbis.so.0 (0x00007f5706734000)
        libogg.so.0 => /tmp/tmp.lVQZh5gl96/libogg.so.0 (0x00007f570652d000)
        libtdb.so.1 => /tmp/tmp.lVQZh5gl96/libtdb.so.1 (0x00007f5706316000)
        libltdl.so.7 => /tmp/tmp.lVQZh5gl96/libltdl.so.7 (0x00007f570610c000)
        libxcb.so.1 => /tmp/tmp.lVQZh5gl96/libxcb.so.1 (0x00007f5705ee4000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f5705ce0000)
        libgmodule-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libgmodule-2.0.so.0 (0x00007f5705adc000)
        libXi.so.6 => /tmp/tmp.lVQZh5gl96/libXi.so.6 (0x00007f57058cc000)
        libXfixes.so.3 => /tmp/tmp.lVQZh5gl96/libXfixes.so.3 (0x00007f57056c6000)
        libatk-bridge-2.0.so.0 => /tmp/tmp.lVQZh5gl96/libatk-bridge-2.0.so.0 (0x00007f5705497000)
        libepoxy.so.0 => /tmp/tmp.lVQZh5gl96/libepoxy.so.0 (0x00007f57051a2000)
        libpangoft2-1.0.so.0 => /tmp/tmp.lVQZh5gl96/libpangoft2-1.0.so.0 (0x00007f5704f8c000)
        libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007f5704d41000)
        libXinerama.so.1 => /tmp/tmp.lVQZh5gl96/libXinerama.so.1 (0x00007f5704b3e000)
        libXrandr.so.2 => /tmp/tmp.lVQZh5gl96/libXrandr.so.2 (0x00007f5704933000)
        libXcursor.so.1 => /tmp/tmp.lVQZh5gl96/libXcursor.so.1 (0x00007f5704728000)
        libXcomposite.so.1 => /tmp/tmp.lVQZh5gl96/libXcomposite.so.1 (0x00007f5704525000)
        libXdamage.so.1 => /tmp/tmp.lVQZh5gl96/libXdamage.so.1 (0x00007f5704322000)
        libxkbcommon.so.0 => /tmp/tmp.lVQZh5gl96/libxkbcommon.so.0 (0x00007f57040e2000)
        libwayland-cursor.so.0 => /tmp/tmp.lVQZh5gl96/libwayland-cursor.so.0 (0x00007f5703eda000)
        libwayland-egl.so.1 => /tmp/tmp.lVQZh5gl96/libwayland-egl.so.1 (0x00007f5703cd8000)
        libwayland-client.so.0 => /tmp/tmp.lVQZh5gl96/libwayland-client.so.0 (0x00007f5703ac9000)
        libXext.so.6 => /tmp/tmp.lVQZh5gl96/libXext.so.6 (0x00007f57038b7000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f57036ae000)
        libffi.so.6 => /tmp/tmp.lVQZh5gl96/libffi.so.6 (0x00007f57034a6000)
        libpcre.so.1 => /tmp/tmp.lVQZh5gl96/libpcre.so.1 (0x00007f5703244000)
        libselinux.so.1 => /tmp/tmp.lVQZh5gl96/libselinux.so.1 (0x00007f570301d000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5702e05000)
        libmount.so.1 => /tmp/tmp.lVQZh5gl96/libmount.so.1 (0x00007f5702bc3000)
        libthai.so.0 => /tmp/tmp.lVQZh5gl96/libthai.so.0 (0x00007f57029b7000)
        libharfbuzz.so.0 => /tmp/tmp.lVQZh5gl96/libharfbuzz.so.0 (0x00007f5702754000)
        libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007f5702478000)
        libpixman-1.so.0 => /tmp/tmp.lVQZh5gl96/libpixman-1.so.0 (0x00007f57021cf000)
        libEGL.so.1 => /tmp/tmp.lVQZh5gl96/libEGL.so.1 (0x00007f5701f99000)
        libxcb-shm.so.0 => /tmp/tmp.lVQZh5gl96/libxcb-shm.so.0 (0x00007f5701d95000)
        libxcb-render.so.0 => /tmp/tmp.lVQZh5gl96/libxcb-render.so.0 (0x00007f5701b87000)
        libXrender.so.1 => /tmp/tmp.lVQZh5gl96/libXrender.so.1 (0x00007f570197c000)
        libGL.so.1 => /usr/lib/libGL.so.1 (0x00007f57016fc000)
        liblzma.so.5 => /tmp/tmp.lVQZh5gl96/liblzma.so.5 (0x00007f57014d6000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f570db99000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f570129e000)
        libXau.so.6 => /tmp/tmp.lVQZh5gl96/libXau.so.6 (0x00007f570109a000)
        libatspi.so.0 => /tmp/tmp.lVQZh5gl96/libatspi.so.0 (0x00007f5700e6a000)
        libdbus-1.so.3 => /usr/lib/libdbus-1.so.3 (0x00007f5700c16000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f5700a0e000)
        libblkid.so.1 => /tmp/tmp.lVQZh5gl96/libblkid.so.1 (0x00007f57007cf000)
        libgraphite2.so.3 => /tmp/tmp.lVQZh5gl96/libgraphite2.so.3 (0x00007f57005a1000)
        libbz2.so.1 => /tmp/tmp.lVQZh5gl96/libbz2.so.1 (0x00007f5700391000)
        libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007f5700153000)
        libxcb-dri2.so.0 => /tmp/tmp.lVQZh5gl96/libxcb-dri2.so.0 (0x00007f56fff4e000)
        libX11-xcb.so.1 => /tmp/tmp.lVQZh5gl96/libX11-xcb.so.1 (0x00007f56ffd4c000)
        libxcb-dri3.so.0 => /usr/lib/libxcb-dri3.so.0 (0x00007f56ffb47000)
        libxcb-xfixes.so.0 => /tmp/tmp.lVQZh5gl96/libxcb-xfixes.so.0 (0x00007f56ff93f000)
        libxcb-present.so.0 => /tmp/tmp.lVQZh5gl96/libxcb-present.so.0 (0x00007f56ff73c000)
        libxcb-sync.so.1 => /tmp/tmp.lVQZh5gl96/libxcb-sync.so.1 (0x00007f56ff535000)
        libxshmfence.so.1 => /tmp/tmp.lVQZh5gl96/libxshmfence.so.1 (0x00007f56ff332000)
        libgbm.so.1 => /tmp/tmp.lVQZh5gl96/libgbm.so.1 (0x00007f56ff124000)
        libwayland-server.so.0 => /tmp/tmp.lVQZh5gl96/libwayland-server.so.0 (0x00007f56fef11000)
        libdrm.so.2 => /usr/lib/libdrm.so.2 (0x00007f56fecfd000)
        libglapi.so.0 => /tmp/tmp.lVQZh5gl96/libglapi.so.0 (0x00007f56feacd000)
        libxcb-glx.so.0 => /tmp/tmp.lVQZh5gl96/libxcb-glx.so.0 (0x00007f56fe8b2000)
        libsystemd.so.0 => /tmp/tmp.lVQZh5gl96/libsystemd.so.0 (0x00007f56fe681000)
        libcap.so.2 => /tmp/tmp.lVQZh5gl96/libcap.so.2 (0x00007f56fe47c000)
        liblz4.so.1 => /tmp/tmp.lVQZh5gl96/liblz4.so.1 (0x00007f56fe267000)
        libgcrypt.so.11 => /tmp/tmp.lVQZh5gl96/libgcrypt.so.11 (0x00007f56fdfe6000)
        libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x00007f56fddc1000)
        libdw.so.1 => /tmp/tmp.lVQZh5gl96/libdw.so.1 (0x00007f56fdb7a000)
        libattr.so.1 => /tmp/tmp.lVQZh5gl96/libattr.so.1 (0x00007f56fd975000)
        libelf.so.1 => /tmp/tmp.lVQZh5gl96/libelf.so.1 (0x00007f56fd75d000)

==========

LD_LIBRARY_PATH: /tmp/tmp.lVQZh5gl96:/usr/lib::/tmp/.mount_RawThek3vKcP/usr/lib/gdk-pixbuf-2.0/loaders
DIRNAME: /tmp/.mount_RawThek3vKcP/usr/bin
DESKTOPFILE_NAME: rawtherapee.desktop
APP_FULL: RawTherapee
APP: RawTherapee
APP_FULL: RawTherapee
APP: RawTherapee
BIN: /tmp/.mount_RawThek3vKcP/usr/bin/rawtherapee
DESKTOPINTEGRATION:
HERE!!!
is_external_process: bundled process found: /tmp/.mount_RawThek3vKcP/usr/bin/zenity
Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.
yesno finished
is_external_process: bundled process found: /tmp/.mount_RawThek3vKcP/usr/bin/zenity
Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.
yesno finished
is_external_process: bundled process found: /tmp/.mount_RawThek3vKcP/usr/bin/rawtherapee

Next, I tried to run --enable-auto-update but that failed, both in the corrupt and the intact AppImages:

morgan@sabayon ~/downloads $ ./RawTherapee-dev-5.4-922-g404ff5e_corrupt.AppImage --enable-auto-update
touch: cannot touch '/home/morgan/.local/appimages/RawTherapee-dev-enable-update': No such file or directory
morgan@sabayon ~/downloads $ ./RawTherapee-dev-5.4-922-g404ff5e.AppImage --enable-auto-update
touch: cannot touch '/home/morgan/.local/appimages/RawTherapee-dev-enable-update': No such file or directory
aferrero2707 commented 6 years ago

@Beep6581

I corrupted it and ran it. It ran successfully. The AppImage did not detect that it was corrupted

This is not how it works... the AppImages do not check themselves. Instead, during the update process the AppImage X downloads and checks the X+1 package before running it. This assumes that X is not corrupted, and that therefore it can reliably check the validity of X+1.

If you think about it, it would be likely that an infected AppImage would be modified to return a false negative when asked to check itself. That's why the AppImages are not self-checking. Also, the check relies on the sha256sum command from the host system (usually located in /usr/bin), which is assumed to be trustworthy.

To check your corrupted AppImage, you should download the corresponding *.sha256sum from github and put it in the same folder as the corrupted AppImage. Then run sha256sum: it should report a checksum mismatch.

Next, I tried to run --enable-auto-update but that failed

My mistake, the $HOME/.local/appimages/ folder is not created when missing. I am updating the AppImages with this fix, meanwhile you could just do

mkdir -p "$HOME/.local/appimages/"

Thanks for checking!

Beep6581 commented 6 years ago

If you think about it, it would be likely that an infected AppImage would be modified to return a false negative when asked to check itself.

Self-checking using a hash would be pointless if the goal was to identify infection, but the goal was to identify a corrupted download, which is far more likely than infection. Since a sidecar file with the reference SHA-256 hash is available, this is possible. However, I'm not requesting that feature, just pointing out what my test was about.

I will test the new AppImage in the evening.

aferrero2707 commented 5 years ago

@Beep6581 some thoughts about a better way to integrate AppImage self-updates... There is a tool called appimageupdatetool that allows to nicely automate the update process, and uses zsync-based incremental updates in order to save bandwidth and fetch only the bits needed to go from one version of the package to another.

Here is an introduction to this tool: https://github.com/AppImage/AppImageUpdate/wiki/Example:-Step-by-step-walkthrough-with-%60appimageupdatetool%60

Would you agree if I try to implement support for this tool in the existing RT AppImages?

Beep6581 commented 5 years ago

Hey @aferrero2707 , I won't be able to take a look in depth until next week, could you ping me then? Then again, if the script does not complicate things for people who don't want to use it, and is known to perform well and be safe to run, then just go ahead.

fliker09 commented 5 years ago

Sorry if off-topic, but how to get CLI version when using AppImage?

Beep6581 commented 5 years ago

Hi @aferrero2707 , I have some free time again. What is the status of the auto-updated integration?

Beep6581 commented 4 years ago

We now have CI for AppImage and Windows builds.