Automated AppImage and Windows builds with Travis CI

aferrero2707 commented 6 years ago

Dear all, I have just finished setting up a GitHub repository to automate the building and packaging of RT for Linux (in AppImage format) and Windows 64-bit (cross-compiled with mingw-w64). The build process is based on Travis CI, and uses custom Docker containers that provide the required dependencies. The packages are automatically uploaded back to GitHub.

At the moment the building is limited to the devRT branch, but other branches (like releases and newlocallab) can be easily added. The build process can be triggered by hand or run regularly (daily or weekly) using cron jobs.

It should also be possible to include OSX packages in the automated building process.

The advantage of this approach is that each build is executed in a well-defined and encapsulated environment, and is perfectly reproducible. Moreover, any user that is added as a collaborator to the github repository should also be able to trigger new builds on Travis, without the need of having a windows system at hand...

First of all, and before doing further developments of the build environment, I'd like to know if the RT developers are interested in this possibility. Of course I'm open to any suggestion and remark!

Beep6581 commented 6 years ago

@aferrero2707 yes it is, and yes I can run it as normal user when I write the full path.

$ PATH="${PATH}:/sbin" ./RawTherapee-dev-5.4-686-g909b76e.AppImage             
AILIBDIR=/tmp/ailibdir.hbyO
deleting /tmp/.mount_T7MWo1/usr/lib/libxcb-dri3.so*
rm: cannot remove '/tmp/.mount_T7MWo1/usr/lib/libxcb-dri3.so.0': Read-only file system
System stdc++ library: "/usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libstdc++.so.6"
System stdc++ library version: "3.4.22"
Bundled stdc++ library version: "3.4.22"
Newest stdc++ library version: "3.4.22"
Using system stdc++ library
fcv: "1.10.1"  fcv2: "1.7.0"
echo "1.10.1 1.7.0" | tr " " "\n" | sort -V | tail -n 1
Newest fontconfig library version: "1.10.1"
Using system fontconfig library
        linux-vdso.so.1 (0x00007ffce77ac000)
        libcanberra-gtk3.so.0 => /tmp/ailibdir.hbyO/libcanberra-gtk3.so.0 (0x00007f6bbfa9f000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0x00007f6bbf748000)
        libcanberra.so.0 => /tmp/ailibdir.hbyO/libcanberra.so.0 (0x00007f6bbf537000)
        libgtk-3.so.0 => /tmp/ailibdir.hbyO/libgtk-3.so.0 (0x00007f6bbec24000)
        libgdk-3.so.0 => /tmp/ailibdir.hbyO/libgdk-3.so.0 (0x00007f6bbe938000)
        libatk-1.0.so.0 => /tmp/ailibdir.hbyO/libatk-1.0.so.0 (0x00007f6bbe712000)
        libgio-2.0.so.0 => /tmp/ailibdir.hbyO/libgio-2.0.so.0 (0x00007f6bbe377000)
        libpangocairo-1.0.so.0 => /tmp/ailibdir.hbyO/libpangocairo-1.0.so.0 (0x00007f6bbe16a000)
        libgdk_pixbuf-2.0.so.0 => /tmp/ailibdir.hbyO/libgdk_pixbuf-2.0.so.0 (0x00007f6bbdf42000)
        libcairo-gobject.so.2 => /tmp/ailibdir.hbyO/libcairo-gobject.so.2 (0x00007f6bbdd39000)
        libpango-1.0.so.0 => /tmp/ailibdir.hbyO/libpango-1.0.so.0 (0x00007f6bbdaee000)
        libcairo.so.2 => /tmp/ailibdir.hbyO/libcairo.so.2 (0x00007f6bbd7c6000)
        libgobject-2.0.so.0 => /tmp/ailibdir.hbyO/libgobject-2.0.so.0 (0x00007f6bbd575000)
        libglib-2.0.so.0 => /tmp/ailibdir.hbyO/libglib-2.0.so.0 (0x00007f6bbd261000)
        libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007f6bbd032000)
        libfftw3f.so.3 => /tmp/ailibdir.hbyO/libfftw3f.so.3 (0x00007f6bbccb9000)
        libfftw3f_omp.so.3 => /tmp/ailibdir.hbyO/libfftw3f_omp.so.3 (0x00007f6bbcab2000)
        libgiomm-2.4.so.1 => /tmp/ailibdir.hbyO/libgiomm-2.4.so.1 (0x00007f6bbc6f4000)
        libglibmm-2.4.so.1 => /tmp/ailibdir.hbyO/libglibmm-2.4.so.1 (0x00007f6bbc474000)
        libsigc-2.0.so.0 => /tmp/ailibdir.hbyO/libsigc-2.0.so.0 (0x00007f6bbc26d000)
        libgthread-2.0.so.0 => /tmp/ailibdir.hbyO/libgthread-2.0.so.0 (0x00007f6bbc06b000)
        libgtkmm-3.0.so.1 => /tmp/ailibdir.hbyO/libgtkmm-3.0.so.1 (0x00007f6bbb8bd000)
        libatkmm-1.6.so.1 => /tmp/ailibdir.hbyO/libatkmm-1.6.so.1 (0x00007f6bbb66f000)
        libgdkmm-3.0.so.1 => /tmp/ailibdir.hbyO/libgdkmm-3.0.so.1 (0x00007f6bbb41b000)
        libpangomm-1.4.so.1 => /tmp/ailibdir.hbyO/libpangomm-1.4.so.1 (0x00007f6bbb1ed000)
        libcairomm-1.0.so.1 => /tmp/ailibdir.hbyO/libcairomm-1.0.so.1 (0x00007f6bbafc7000)
        libiptcdata.so.0 => /tmp/ailibdir.hbyO/libiptcdata.so.0 (0x00007f6bbadbb000)
        libjpeg.so.62 => /tmp/ailibdir.hbyO/libjpeg.so.62 (0x00007f6bbab66000)
        liblcms2.so.2 => /tmp/ailibdir.hbyO/liblcms2.so.2 (0x00007f6bba90c000)
        libpng15.so.15 => /tmp/ailibdir.hbyO/libpng15.so.15 (0x00007f6bba6e1000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f6bba4c8000)
        libtiff.so.5 => /tmp/ailibdir.hbyO/libtiff.so.5 (0x00007f6bba251000)
        liblensfun.so.1 => /tmp/ailibdir.hbyO/liblensfun.so.1 (0x00007f6bba032000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f6bb9cde000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libstdc++.so.6 (0x00007f6bb98ae000)
        libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libgomp.so.1 (0x00007f6bb9679000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libgcc_s.so.1 (0x00007f6bb9462000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6bb9241000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f6bb8e6c000)
        libvorbisfile.so.3 => /tmp/ailibdir.hbyO/libvorbisfile.so.3 (0x00007f6bb8c63000)
        libvorbis.so.0 => /tmp/ailibdir.hbyO/libvorbis.so.0 (0x00007f6bb8a36000)
        libogg.so.0 => /tmp/ailibdir.hbyO/libogg.so.0 (0x00007f6bb882f000)
        libtdb.so.1 => /tmp/ailibdir.hbyO/libtdb.so.1 (0x00007f6bb8618000)
        libltdl.so.7 => /tmp/ailibdir.hbyO/libltdl.so.7 (0x00007f6bb840e000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007f6bb81dd000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f6bb7fd9000)
        libgmodule-2.0.so.0 => /tmp/ailibdir.hbyO/libgmodule-2.0.so.0 (0x00007f6bb7dd5000)
        libXi.so.6 => /tmp/ailibdir.hbyO/libXi.so.6 (0x00007f6bb7bc5000)
        libXfixes.so.3 => /tmp/ailibdir.hbyO/libXfixes.so.3 (0x00007f6bb79bf000)
        libatk-bridge-2.0.so.0 => /tmp/ailibdir.hbyO/libatk-bridge-2.0.so.0 (0x00007f6bb7790000)
        libepoxy.so.0 => /tmp/ailibdir.hbyO/libepoxy.so.0 (0x00007f6bb749b000)
        libpangoft2-1.0.so.0 => /tmp/ailibdir.hbyO/libpangoft2-1.0.so.0 (0x00007f6bb7285000)
        libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007f6bb7039000)
        libXinerama.so.1 => /tmp/ailibdir.hbyO/libXinerama.so.1 (0x00007f6bb6e36000)
        libXrandr.so.2 => /tmp/ailibdir.hbyO/libXrandr.so.2 (0x00007f6bb6c2b000)
        libXcursor.so.1 => /tmp/ailibdir.hbyO/libXcursor.so.1 (0x00007f6bb6a20000)
        libXcomposite.so.1 => /tmp/ailibdir.hbyO/libXcomposite.so.1 (0x00007f6bb681d000)
        libXdamage.so.1 => /tmp/ailibdir.hbyO/libXdamage.so.1 (0x00007f6bb661a000)
        libxkbcommon.so.0 => /tmp/ailibdir.hbyO/libxkbcommon.so.0 (0x00007f6bb63da000)
        libwayland-cursor.so.0 => /tmp/ailibdir.hbyO/libwayland-cursor.so.0 (0x00007f6bb61d2000)
        libwayland-egl.so.1 => /tmp/ailibdir.hbyO/libwayland-egl.so.1 (0x00007f6bb5fd0000)
        libwayland-client.so.0 => /usr/lib/libwayland-client.so.0 (0x00007f6bb5dbf000)
        libXext.so.6 => /tmp/ailibdir.hbyO/libXext.so.6 (0x00007f6bb5bad000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f6bb59a4000)
        libffi.so.6 => /tmp/ailibdir.hbyO/libffi.so.6 (0x00007f6bb579c000)
        libpcre.so.1 => /tmp/ailibdir.hbyO/libpcre.so.1 (0x00007f6bb553a000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f6bb530d000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f6bb50f5000)
        libmount.so.1 => /tmp/ailibdir.hbyO/libmount.so.1 (0x00007f6bb4eb3000)
        libthai.so.0 => /tmp/ailibdir.hbyO/libthai.so.0 (0x00007f6bb4ca7000)
        libharfbuzz.so.0 => /tmp/ailibdir.hbyO/libharfbuzz.so.0 (0x00007f6bb4a44000)
        libfreetype.so.6 => /tmp/ailibdir.hbyO/libfreetype.so.6 (0x00007f6bb479e000)
        libpixman-1.so.0 => /tmp/ailibdir.hbyO/libpixman-1.so.0 (0x00007f6bb44f5000)
        libEGL.so.1 => /usr/lib/libEGL.so.1 (0x00007f6bb42b6000)
        libxcb-shm.so.0 => /tmp/ailibdir.hbyO/libxcb-shm.so.0 (0x00007f6bb40b2000)
        libxcb-render.so.0 => /tmp/ailibdir.hbyO/libxcb-render.so.0 (0x00007f6bb3ea4000)
        libXrender.so.1 => /tmp/ailibdir.hbyO/libXrender.so.1 (0x00007f6bb3c99000)
        libGL.so.1 => /usr/lib/libGL.so.1 (0x00007f6bb3a1a000)
        liblzma.so.5 => /tmp/ailibdir.hbyO/liblzma.so.5 (0x00007f6bb37f4000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6bbfca4000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f6bb35bc000)
        libXau.so.6 => /tmp/ailibdir.hbyO/libXau.so.6 (0x00007f6bb33b8000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007f6bb31b1000)
        libatspi.so.0 => /tmp/ailibdir.hbyO/libatspi.so.0 (0x00007f6bb2f81000)
        libdbus-1.so.3 => /tmp/ailibdir.hbyO/libdbus-1.so.3 (0x00007f6bb2d31000)
        libblkid.so.1 => /tmp/ailibdir.hbyO/libblkid.so.1 (0x00007f6bb2af2000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f6bb28ea000)
        libgraphite2.so.3 => /tmp/ailibdir.hbyO/libgraphite2.so.3 (0x00007f6bb26bc000)
        libxcb-dri2.so.0 => /tmp/ailibdir.hbyO/libxcb-dri2.so.0 (0x00007f6bb24b7000)
        libX11-xcb.so.1 => /tmp/ailibdir.hbyO/libX11-xcb.so.1 (0x00007f6bb22b5000)
        libxcb-dri3.so.0 => /tmp/ailibdir.hbyO/libxcb-dri3.so.0 (0x00007f6bb20b2000)
        libxcb-xfixes.so.0 => /tmp/ailibdir.hbyO/libxcb-xfixes.so.0 (0x00007f6bb1eaa000)
        libxcb-present.so.0 => /tmp/ailibdir.hbyO/libxcb-present.so.0 (0x00007f6bb1ca7000)
        libxcb-sync.so.1 => /tmp/ailibdir.hbyO/libxcb-sync.so.1 (0x00007f6bb1aa0000)
        libxshmfence.so.1 => /tmp/ailibdir.hbyO/libxshmfence.so.1 (0x00007f6bb189d000)
        libgbm.so.1 => /usr/lib/libgbm.so.1 (0x00007f6bb168e000)
        libwayland-server.so.0 => /usr/lib/libwayland-server.so.0 (0x00007f6bb1478000)
        libdrm.so.2 => /usr/lib/libdrm.so.2 (0x00007f6bb1264000)
        libglapi.so.0 => /tmp/ailibdir.hbyO/libglapi.so.0 (0x00007f6bb1034000)
        libxcb-glx.so.0 => /tmp/ailibdir.hbyO/libxcb-glx.so.0 (0x00007f6bb0e19000)
        libXxf86vm.so.1 => /tmp/ailibdir.hbyO/libXxf86vm.so.1 (0x00007f6bb0c13000)
        libbsd.so.0 => /usr/lib/libbsd.so.0 (0x00007f6bb09fc000)
        libsystemd.so.0 => /tmp/ailibdir.hbyO/libsystemd.so.0 (0x00007f6bb07cb000)
        libcap.so.2 => /tmp/ailibdir.hbyO/libcap.so.2 (0x00007f6bb05c6000)
        liblz4.so.1 => /tmp/ailibdir.hbyO/liblz4.so.1 (0x00007f6bb03b1000)
        libgcrypt.so.11 => /tmp/ailibdir.hbyO/libgcrypt.so.11 (0x00007f6bb0130000)
        libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x00007f6baff0b000)
        libdw.so.1 => /tmp/ailibdir.hbyO/libdw.so.1 (0x00007f6bafcc4000)
        libattr.so.1 => /tmp/ailibdir.hbyO/libattr.so.1 (0x00007f6bafabf000)
        libelf.so.1 => /tmp/ailibdir.hbyO/libelf.so.1 (0x00007f6baf8a7000)
        libbz2.so.1 => /tmp/ailibdir.hbyO/libbz2.so.1 (0x00007f6baf697000)
/tmp/.mount_T7MWo1/usr/bin/rawtherapee: symbol lookup error: /usr/lib/libGL.so.1: undefined symbol: xcb_dri3_get_supported_modifiers

aferrero2707 commented 6 years ago

@Beep6581 sorry, I still had at lest one mistake in the new scripts, causing this error: rm: cannot remove '/tmp/.mount_T7MWo1/usr/lib/libxcb-dri3.so.0': Read-only file system

For your info, the idea is to disable the bundled libxcb-dri3.so.0 library, if the one provided by the system is newer...

Could you try this new package, still manually adding /sbin to your PATH?

Thanks!!!

Beep6581 commented 6 years ago

@aferrero2707 success, it worked!

$ PATH="${PATH}:/sbin" ./RawTherapee-dev-5.4-690-g9847285.AppImage 
AILIBDIR=/tmp/ailibdir.maU1
deleting /tmp/ailibdir.maU1/libxcb-dri3.so*
System stdc++ library: "/usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libstdc++.so.6"
System stdc++ library version: "3.4.22"
Bundled stdc++ library version: "3.4.22"
Newest stdc++ library version: "3.4.22"
Using system stdc++ library
fcv: "1.10.1"  fcv2: "1.7.0"
echo "1.10.1 1.7.0" | tr " " "\n" | sort -V | tail -n 1
Newest fontconfig library version: "1.10.1"
Using system fontconfig library
        linux-vdso.so.1 (0x00007ffe726e2000)
        libcanberra-gtk3.so.0 => /tmp/ailibdir.maU1/libcanberra-gtk3.so.0 (0x00007fe33e35c000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0x00007fe33e005000)
        libcanberra.so.0 => /tmp/ailibdir.maU1/libcanberra.so.0 (0x00007fe33ddf4000)
        libgtk-3.so.0 => /tmp/ailibdir.maU1/libgtk-3.so.0 (0x00007fe33d4e1000)
        libgdk-3.so.0 => /tmp/ailibdir.maU1/libgdk-3.so.0 (0x00007fe33d1f5000)
        libatk-1.0.so.0 => /tmp/ailibdir.maU1/libatk-1.0.so.0 (0x00007fe33cfcf000)
        libgio-2.0.so.0 => /tmp/ailibdir.maU1/libgio-2.0.so.0 (0x00007fe33cc34000)
        libpangocairo-1.0.so.0 => /tmp/ailibdir.maU1/libpangocairo-1.0.so.0 (0x00007fe33ca27000)
        libgdk_pixbuf-2.0.so.0 => /tmp/ailibdir.maU1/libgdk_pixbuf-2.0.so.0 (0x00007fe33c7ff000)
        libcairo-gobject.so.2 => /tmp/ailibdir.maU1/libcairo-gobject.so.2 (0x00007fe33c5f6000)
        libpango-1.0.so.0 => /tmp/ailibdir.maU1/libpango-1.0.so.0 (0x00007fe33c3ab000)
        libcairo.so.2 => /tmp/ailibdir.maU1/libcairo.so.2 (0x00007fe33c083000)
        libgobject-2.0.so.0 => /tmp/ailibdir.maU1/libgobject-2.0.so.0 (0x00007fe33be32000)
        libglib-2.0.so.0 => /tmp/ailibdir.maU1/libglib-2.0.so.0 (0x00007fe33bb1e000)
        libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007fe33b8ef000)
        libfftw3f.so.3 => /tmp/ailibdir.maU1/libfftw3f.so.3 (0x00007fe33b576000)
        libfftw3f_omp.so.3 => /tmp/ailibdir.maU1/libfftw3f_omp.so.3 (0x00007fe33b36f000)
        libgiomm-2.4.so.1 => /tmp/ailibdir.maU1/libgiomm-2.4.so.1 (0x00007fe33afb1000)
        libglibmm-2.4.so.1 => /tmp/ailibdir.maU1/libglibmm-2.4.so.1 (0x00007fe33ad31000)
        libsigc-2.0.so.0 => /tmp/ailibdir.maU1/libsigc-2.0.so.0 (0x00007fe33ab2a000)
        libgthread-2.0.so.0 => /tmp/ailibdir.maU1/libgthread-2.0.so.0 (0x00007fe33a928000)
        libgtkmm-3.0.so.1 => /tmp/ailibdir.maU1/libgtkmm-3.0.so.1 (0x00007fe33a17a000)
        libatkmm-1.6.so.1 => /tmp/ailibdir.maU1/libatkmm-1.6.so.1 (0x00007fe339f2c000)
        libgdkmm-3.0.so.1 => /tmp/ailibdir.maU1/libgdkmm-3.0.so.1 (0x00007fe339cd8000)
        libpangomm-1.4.so.1 => /tmp/ailibdir.maU1/libpangomm-1.4.so.1 (0x00007fe339aaa000)
        libcairomm-1.0.so.1 => /tmp/ailibdir.maU1/libcairomm-1.0.so.1 (0x00007fe339884000)
        libiptcdata.so.0 => /tmp/ailibdir.maU1/libiptcdata.so.0 (0x00007fe339678000)
        libjpeg.so.62 => /tmp/ailibdir.maU1/libjpeg.so.62 (0x00007fe339423000)
        liblcms2.so.2 => /tmp/ailibdir.maU1/liblcms2.so.2 (0x00007fe3391c9000)
        libpng15.so.15 => /tmp/ailibdir.maU1/libpng15.so.15 (0x00007fe338f9e000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fe338d85000)
        libtiff.so.5 => /tmp/ailibdir.maU1/libtiff.so.5 (0x00007fe338b0e000)
        liblensfun.so.1 => /tmp/ailibdir.maU1/liblensfun.so.1 (0x00007fe3388ef000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fe33859b000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libstdc++.so.6 (0x00007fe33816b000)
        libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libgomp.so.1 (0x00007fe337f36000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/6.4.0/libgcc_s.so.1 (0x00007fe337d1f000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe337afe000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fe337729000)
        libvorbisfile.so.3 => /tmp/ailibdir.maU1/libvorbisfile.so.3 (0x00007fe337520000)
        libvorbis.so.0 => /tmp/ailibdir.maU1/libvorbis.so.0 (0x00007fe3372f3000)
        libogg.so.0 => /tmp/ailibdir.maU1/libogg.so.0 (0x00007fe3370ec000)
        libtdb.so.1 => /tmp/ailibdir.maU1/libtdb.so.1 (0x00007fe336ed5000)
        libltdl.so.7 => /tmp/ailibdir.maU1/libltdl.so.7 (0x00007fe336ccb000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007fe336a9a000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fe336896000)
        libgmodule-2.0.so.0 => /tmp/ailibdir.maU1/libgmodule-2.0.so.0 (0x00007fe336692000)
        libXi.so.6 => /tmp/ailibdir.maU1/libXi.so.6 (0x00007fe336482000)
        libXfixes.so.3 => /tmp/ailibdir.maU1/libXfixes.so.3 (0x00007fe33627c000)
        libatk-bridge-2.0.so.0 => /tmp/ailibdir.maU1/libatk-bridge-2.0.so.0 (0x00007fe33604d000)
        libepoxy.so.0 => /tmp/ailibdir.maU1/libepoxy.so.0 (0x00007fe335d58000)
        libpangoft2-1.0.so.0 => /tmp/ailibdir.maU1/libpangoft2-1.0.so.0 (0x00007fe335b42000)
        libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007fe3358f6000)
        libXinerama.so.1 => /tmp/ailibdir.maU1/libXinerama.so.1 (0x00007fe3356f3000)
        libXrandr.so.2 => /tmp/ailibdir.maU1/libXrandr.so.2 (0x00007fe3354e8000)
        libXcursor.so.1 => /tmp/ailibdir.maU1/libXcursor.so.1 (0x00007fe3352dd000)
        libXcomposite.so.1 => /tmp/ailibdir.maU1/libXcomposite.so.1 (0x00007fe3350da000)
        libXdamage.so.1 => /tmp/ailibdir.maU1/libXdamage.so.1 (0x00007fe334ed7000)
        libxkbcommon.so.0 => /tmp/ailibdir.maU1/libxkbcommon.so.0 (0x00007fe334c97000)
        libwayland-cursor.so.0 => /tmp/ailibdir.maU1/libwayland-cursor.so.0 (0x00007fe334a8f000)
        libwayland-egl.so.1 => /tmp/ailibdir.maU1/libwayland-egl.so.1 (0x00007fe33488d000)
        libwayland-client.so.0 => /usr/lib/libwayland-client.so.0 (0x00007fe33467c000)
        libXext.so.6 => /tmp/ailibdir.maU1/libXext.so.6 (0x00007fe33446a000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fe334261000)
        libffi.so.6 => /tmp/ailibdir.maU1/libffi.so.6 (0x00007fe334059000)
        libpcre.so.1 => /tmp/ailibdir.maU1/libpcre.so.1 (0x00007fe333df7000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fe333bca000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fe3339b2000)
        libmount.so.1 => /tmp/ailibdir.maU1/libmount.so.1 (0x00007fe333770000)
        libthai.so.0 => /tmp/ailibdir.maU1/libthai.so.0 (0x00007fe333564000)
        libharfbuzz.so.0 => /tmp/ailibdir.maU1/libharfbuzz.so.0 (0x00007fe333301000)
        libfreetype.so.6 => /tmp/ailibdir.maU1/libfreetype.so.6 (0x00007fe33305b000)
        libpixman-1.so.0 => /tmp/ailibdir.maU1/libpixman-1.so.0 (0x00007fe332db2000)
        libEGL.so.1 => /usr/lib/libEGL.so.1 (0x00007fe332b73000)
        libxcb-shm.so.0 => /tmp/ailibdir.maU1/libxcb-shm.so.0 (0x00007fe33296f000)
        libxcb-render.so.0 => /tmp/ailibdir.maU1/libxcb-render.so.0 (0x00007fe332761000)
        libXrender.so.1 => /tmp/ailibdir.maU1/libXrender.so.1 (0x00007fe332556000)
        libGL.so.1 => /usr/lib/libGL.so.1 (0x00007fe3322d7000)
        liblzma.so.5 => /tmp/ailibdir.maU1/liblzma.so.5 (0x00007fe3320b1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe33e561000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fe331e79000)
        libXau.so.6 => /tmp/ailibdir.maU1/libXau.so.6 (0x00007fe331c75000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007fe331a6e000)
        libatspi.so.0 => /tmp/ailibdir.maU1/libatspi.so.0 (0x00007fe33183e000)
        libdbus-1.so.3 => /tmp/ailibdir.maU1/libdbus-1.so.3 (0x00007fe3315ee000)
        libblkid.so.1 => /tmp/ailibdir.maU1/libblkid.so.1 (0x00007fe3313af000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007fe3311a7000)
        libgraphite2.so.3 => /tmp/ailibdir.maU1/libgraphite2.so.3 (0x00007fe330f79000)
        libxcb-dri2.so.0 => /tmp/ailibdir.maU1/libxcb-dri2.so.0 (0x00007fe330d74000)
        libX11-xcb.so.1 => /tmp/ailibdir.maU1/libX11-xcb.so.1 (0x00007fe330b72000)
        libxcb-dri3.so.0 => /usr/lib/libxcb-dri3.so.0 (0x00007fe33096d000)
        libxcb-xfixes.so.0 => /tmp/ailibdir.maU1/libxcb-xfixes.so.0 (0x00007fe330765000)
        libxcb-present.so.0 => /tmp/ailibdir.maU1/libxcb-present.so.0 (0x00007fe330562000)
        libxcb-sync.so.1 => /tmp/ailibdir.maU1/libxcb-sync.so.1 (0x00007fe33035b000)
        libxshmfence.so.1 => /tmp/ailibdir.maU1/libxshmfence.so.1 (0x00007fe330158000)
        libgbm.so.1 => /usr/lib/libgbm.so.1 (0x00007fe32ff49000)
        libwayland-server.so.0 => /usr/lib/libwayland-server.so.0 (0x00007fe32fd33000)
        libdrm.so.2 => /usr/lib/libdrm.so.2 (0x00007fe32fb1f000)
        libglapi.so.0 => /tmp/ailibdir.maU1/libglapi.so.0 (0x00007fe32f8ef000)
        libxcb-glx.so.0 => /tmp/ailibdir.maU1/libxcb-glx.so.0 (0x00007fe32f6d4000)
        libXxf86vm.so.1 => /tmp/ailibdir.maU1/libXxf86vm.so.1 (0x00007fe32f4ce000)
        libbsd.so.0 => /usr/lib/libbsd.so.0 (0x00007fe32f2b7000)
        libsystemd.so.0 => /tmp/ailibdir.maU1/libsystemd.so.0 (0x00007fe32f086000)
        libcap.so.2 => /tmp/ailibdir.maU1/libcap.so.2 (0x00007fe32ee81000)
        liblz4.so.1 => /tmp/ailibdir.maU1/liblz4.so.1 (0x00007fe32ec6c000)
        libgcrypt.so.11 => /tmp/ailibdir.maU1/libgcrypt.so.11 (0x00007fe32e9eb000)
        libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x00007fe32e7c6000)
        libdw.so.1 => /tmp/ailibdir.maU1/libdw.so.1 (0x00007fe32e57f000)
        libattr.so.1 => /tmp/ailibdir.maU1/libattr.so.1 (0x00007fe32e37a000)
        libelf.so.1 => /tmp/ailibdir.maU1/libelf.so.1 (0x00007fe32e162000)
        libbz2.so.1 => /tmp/ailibdir.maU1/libbz2.so.1 (0x00007fe32df52000)
Gtk-Message: Failed to load module "canberra-gtk-module"

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:68:35: The style property GtkButton:child-displacement-x is deprecated and shouldn't be used anymore. It will be removed in a future version

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:69:35: The style property GtkButton:child-displacement-y is deprecated and shouldn't be used anymore. It will be removed in a future version

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:70:34: The style property GtkCheckButton:indicator-size is deprecated and shouldn't be used anymore. It will be removed in a future version

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:71:36: The style property GtkCheckMenuItem:indicator-size is deprecated and shouldn't be used anymore. It will be removed in a future version

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:73:46: The style property GtkScrolledWindow:scrollbars-within-bevel is deprecated and shouldn't be used anymore. It will be removed in a future version

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:76:30: The style property GtkExpander:expander-size is deprecated and shouldn't be used anymore. It will be removed in a future version

(rawtherapee:36668): Gtk-WARNING **: Theme parsing error: gtk.css:83:29: The style property GtkStatusbar:shadow-type is deprecated and shouldn't be used anymore. It will be removed in a future version
Could not extract thumb from /home/morgan/photos/test_images/hdrmerge_range_0-1.dng

(rawtherapee:36668): GLib-GIO-WARNING **: Can't find module 'dconf' specified in GSETTINGS_BACKEND
GLib-GIO-Message: Using the 'memory' GSettings backend.  Your settings will not be saved or shared with other applications.

File processed successfully, felt as fast as my native builds.

aferrero2707 commented 6 years ago

felt as fast as my native builds.

This is probably the most important thing! There would be no point in putting so much efforts into the AppImage packages if they would result in sub-optimal performances...

aferrero2707 commented 6 years ago

@Beep6581 I have uploaded a new AppImage package, that should fix all recent issues that have been reported. The package was built under CentOS7, and should be compatible with all mainstream Linux distributions (including older ones like Debian-stable and Ubuntu 14.04).

It would be great if it could be tested by as many Linux users as possible... If it works ok, I will also refresh the appimages for the other branches.

Thanks!

Beep6581 commented 6 years ago

@aferrero2707 great, I confirm that the above file runs well in Sabayon.

I opened a forum post to get it tested: https://discuss.pixls.us/t/test-the-latest-rawtherapee-appimage/8700 I also posted on Google+ https://plus.google.com/b/106783532637761598368/+rawtherapee/posts/eeTzpjVaQJQ

Once confirmed, we need to replace the 5.4 AppImage on the website with a new one built this way, as the one on our website doesn't run on my system.

aferrero2707 commented 6 years ago

I confirm that the above file runs well in Sabayon.

Excellent!!!

Once confirmed, we need to replace the 5.4 AppImage on the website with a new one built this way, as the one on our website doesn't run on my system.

That will be rather easy, can be done together with the other development branches...

aferrero2707 commented 6 years ago

@Beep6581 from the feedback I have seen, it looks like there are no big issues with the new package... what do you think?

Another aspect that I would like to tackle is how to integrate the CLI version of RT. I see two possibilities:

produce a CLI AppImage package - straightforward but heavy in terms of disk storage and network traffic
add a special --cli command-line switch that would instruct the AppImage to invoke the CLI version of RT, passing the rest of the command-line parameters to it

Any opinion?

Beep6581 commented 6 years ago

@aferrero2707 I think we should proceed to replace the 5.4 AppImage, and replace all current dev AppImages with a new one in the forum dev build page, as soon as the CLI issue is solved.

I agree with a --cli switch. That would be far better than wasting electricity, generating heat and using up bandwidth by duplicating AppImage overhead. How do AppImage command-line options work, are they handled by a bash script? If so, https://mywiki.wooledge.org/BashFAQ/035

aferrero2707 commented 6 years ago

I have done some more work on the AppImage package, and a new version is available here: https://github.com/Beep6581/RawTherapee/releases/download/nightly/RawTherapee-dev-5.4-752-g5471b34.AppImage

The main improvements are the following:

the appimage now correctly handles input images with relative paths
it is now possible to run the CLI version of RT by adding the --cli command line switch as first parameter, followed by the usual RT CLI options

Please give this newest version a try. If it works OK, I will recreate all other RT AppImage packages so that official downloads can also be updated.

Thanks!

Beep6581 commented 5 years ago

Hi @aferrero2707 I was looking for a dev build for the person in issue 4752 when I noticed that the latest CI Windows build is from 2017-11-20: https://github.com/aferrero2707/RawTherapee/releases/tag/continuous

How do we trigger a new Windows build?
Do those builds include gdb.exe and a debug rawtherapee.exe?

aferrero2707 commented 5 years ago

@Beep6581 the CI windows builds were stopped some time ago, because there was no general consensus about their usefulness (other users are already providing Win64 RT builds). I also removed most of my Win64 packages to avoid confusion with the official ones...

However, it should be easy to re-start the automated builds if needed, and generated packages for the various development branches. Just let me know.

Beep6581 commented 5 years ago

They certainly are useful, but a Windows user would need to commit to helping you test and tweak the system.

Beep6581 commented 5 years ago

As they're very outdated, I removed the link from the development builds page:

  * https://github.com/aferrero2707/RawTherapee/releases/tag/continuous
    * Builds by @Carmelo_DrRaw, automatically generated from his fork.
    * This automated build system is under development - please report issues.

gaaned92 commented 5 years ago

@aferrero2707 because there was no general consensus about their usefulness (other users are already providing Win64 RT builds).

Even if I regularly provide windows builds, I think automatic builds is the way to go and should be useful also for devs. If you decide to re-start the windows automatic build, I will try to help testing as I did when you began this work. I noticed that some projects (mypaint, Darktable..) use Appveyor to build on windows/MSYS2. Perhaps it could be a good choice. @Beep6581 I will be out of home from september 6 until october 5.

Beep6581 commented 5 years ago

@gaaned92 enjoy :)

SilvioGrosso commented 5 years ago

Hello everyone,

It would be a pleasure for me as well to test these Windows daily builds. Lately, I have tested the RawTherapee AppImage for Linux, on Kubuntu, 18.04 - KDE, through VirtualBox. On Windows, I can test these daily builds on Windows 10 - Windows 8.1. All in all, I suppose most users are going to opt for the Windows versions in the end...

Thanks a lot indeed : the 5.5 upcoming release is going to rock ! :-)

Beep6581 commented 5 years ago

To completely replace humans (and "take their jobs"), the automated Windows builds would need to support the installer script, and include gdb.exe and rawtherapee-debug.exe.

gaaned92 commented 5 years ago

@aferrero2707 @Beep6581 On Appveyor. in addition to Msys2, windowsinnosetup, 7zip (and lot of other sw) are installled. I tried to use it yesterday on the basis of mypaint. I forked RT see https://github.com/gaaned92/RawTherapee and opened a github account on appveyor. I vreated a branch av and two files appveyor.yml and tools/win/appveyor containing the bash scripts. As I am completely non proficient with SW, I was rather slow. Sow now, I got Msys2 updated and completed, libiptcdata built and now beginning to build Rawtherapee. I think what I made is clumsy, but it is a proof of concept. I noticed the following:

free for github users.
automatic build trigger
easy to program (not so for me as I have a lot of things to learn!)
possible to replicate what is done on native windows
easy to build rawtherapee and rawtherapee-debug to include the two
download gdb from the site to include it

That's only my one cent if it can help, and its up to devs (@aferrero2707 ) to decide what to do and program it.

aferrero2707 commented 5 years ago

@gaaned92 thanks for the AppVeyor instructions!

At the moment I have a working cross-compilation environment, based on mingw-w64, that has from my point of view few advantages:

it runs under Linux, using an ad-hoc Docker container. This means that the building+packaging process can be run and tested on any system supporting Docker (Linux/OSX/Windows)
as everything is executed inside the Docker container, it will be extremely easy to move the automated build process to any CI environment that supports Docker - I am using Travis CI for the moment
past tests have shown that there are no major differences with respect to a native Windows build, particularly not performance-wise

At the moment the package is created in the form of a self-contained and portable .zip file. A proper installer is not yet supported. Packaging of the debug version also requires a bit of additional work, but not too much.

The new experimental Win64 package is available here: https://github.com/aferrero2707/rt-win64/releases/download/continuous/rawtherapee-w64-20180831_1419-git-dev.zip I could only test it with wine so far... I spotted few problems with missing fonts, but apart from that it seems to run correctly. @gaaned92 @SilvioGrosso @heckflosse could you please give it a try and let me know?

aferrero2707 commented 5 years ago

@Beep6581 since the latest AppImage packages seem to work properly on most of the distributions, I went on and updated all the packages in the "nightly builds" release. The package for the official 5.4 release is available here.

Later today I will test each package to see if it runs correctly, but a second look would not harm...

Thanks!

Beep6581 commented 5 years ago

RawTherapee-releases-5.4.AppImage works fine here in Sabayon @aferrero2707

Beep6581 commented 5 years ago

I uploaded the AppImage to our website.

SilvioGrosso commented 5 years ago

Hello @aferrero2707,

To start with, THANKS A LOT for this new build :-) 1) Just tested today's new build on Windows 8.1 - 64 bit (build 9600). Rawtherapee starts and It does not crash when working with it. It even looks as snappy as the "official" 5.4 stable version. I mean the one offered currently on the RawTherappe web-site. I have quickly tested its GUIs with my NEF images (Nikon D700). Some icons are missing though.

2) I have also tested the build on Windows 10 - 64 bit (build 17134). Rawtherapee starts and It does not crash when working with it. It does looks as snappy as the "official" 5.4 version. The one offered currently on the RawTherappe web-site. I have quickly tested its GUIs with my NEF images (Nikon D700) Some icons are missing, as it occurs on Windows 8.1 (see above).

All in all, this build looks quite good as far as I have tested it quickly so far :-)

Here is the screenshot on Windows 10. Take a look on the right side for the missing icons: close, minimize software's GUI; plus the buttons to increase-decrease the options related to the different tools + -

https://www.dropbox.com/s/c9el26icabluzbk/RAWTHERAPEE_MISSING_ICONS_WINDOWS_10.png?dl=0

gaaned92 commented 5 years ago

@aferrero2707 Thanks for your build

starts and runs it seems as fast as my builds. Edited and processed a few files without problem :thumbsup:
The window decoration is not a native windows one but seems generated by gtk3. The missing icones should be found in adwaita theme. The clean build hereunder provides the missing icones.
It seems that a lot of files are useless and could be removed

If it can help, you will find your build cleaned with required adwaita icones located at right place at https://keybase.pub/gaaned92/RTW64NightlyBuilds/rawtherapee-w64-20180831_1419-git-dev.zip

Edit: could you name windows build in the same manner as the appimages, for instance RawTherapee-dev-5.4-752-g5471b34.AppImage becomes for window64 builds RawTherapee-dev-5.4-752-g5471b34-WinVista-64.zip

aferrero2707 commented 5 years ago

@gaaned92 I have followed your example and fixed the missing icons, as well as removed part of the unnecessary files. The new package is available here: https://github.com/aferrero2707/rt-win64/releases/download/continuous/rawtherapee-w64-20180901_1924-git-dev.zip

There is still some problem with SVG icons, but the rest seems to work properly... could you have a look?

Meanwhile, I have started to look into the InnoSetup part...

gaaned92 commented 5 years ago

@aferrero2707

warnings from rawtherapee

(rawtherapee.exe:19620): Gdk-WARNING **: 10:06:51.731: Failed to load cursor theme Adwaita

These are not Windows cursors: see hereunder about cursor subdir

(rawtherapee.exe:19620): Gtk-WARNING **: 10:06:52.410: Could not load a pixbuf from /org/gtk/libgtk/theme/Adwaita/assets/check-symbolic.svg.
This may indicate that pixbuf loaders or the mime database could not be found.
ToneMapFattal02: alpha = 1, beta = 0.91, detail_level = 3

Don't understand the problem! the svg loader seems well located. I did'nt succeed to correct the warning.

(rawtherapee.exe:19620): Gtk-WARNING **: 10:06:52.644: Could not find the icon 'window-minimize-symbolic-ltr'. The 'hicolor' theme
was not found either, perhaps you need to install it.
You can get a copy from:
        http://icon-theme.freedesktop.org/releases

I suppressed hicolor as it is empty. I cannot find this icon in the adwaita/ 16x16 /action . Does'nt complain and display icons with my adwaita dir!

Some remarks regarding the subdirs:

share : you only need glib-2.0, lensfun and icons.
share/icons/Advaita/cursors: the cursors are not windows cursors with a .cur suffix. The buggy content is about the same times ago in MSYS2(there was a ticket in msys2). Perhaps adding the .cur suffix correct it but I doubt.
share/icons/hicolor: not needed
share/ icons /adwaita: a very small part of the icons is needed. scalable and scalable-up-to-32 not needed.

hereunder how I populate it in MSYS2 with minimal required icons

# Icons
        listedirs=" icons/Adwaita/16x16/actions
          icons/Adwaita/16x16/devices icons/Adwaita/16x16/mimetypes
          icons/Adwaita/16x16/places icons/Adwaita/16x16/status 
          icons/Adwaita/48x48/devices  "
        for v in $listedirs
        do
          echo $v
          robocopy /mingw$BD/share/$v $destination/share/$v -s 1>/dev/null
        done
        robocopy /mingw$BD/share/icons/adwaita/24x24/status $destination/share/icons/adwaita/24x24/status image-missing.png 1>/dev/null
#cursors
        robocopy  /mingw$BD/share/icons/Adwaita/cursors    $destination/share/icons/Adwaita/cursors plus.cur \
        sb_h_double_arrow.cur sb_left_arrow.cur sb_right_arrow.cur sb_v_double_arrow.cur 1>/dev/null
#Adwaita files
        robocopy  /mingw$BD/share/icons/Adwaita    $destination/share/icons/Adwaita \
        icon-theme.cache index.theme 1>/dev/null

missing icons: there are still missing icons at top right of main window. If I replace the adwaita subdir by my own adwaita subdir its ok as it contains the missing icons.
lib : gtk-2.0 not needed.
lib/gdk-pixbuf-2.0/2.10.0: Perhaps it should contain loaders.cache.

#gdk-pixbuf     
        robocopy /mingw$BD/lib/gdk-pixbuf-2.0 $destination/lib/gdk-pixbuf-2.0 -s 1>/dev/null

Innosetup: were you able to generate windowsinnosetup.iss as it is not present in your build

aferrero2707 commented 5 years ago

@gaaned92 thanks for your suggestions! I [repared a new package, available here.

cursors: there was a mistake in my build scripts, and adwaita was compiled for a Linux host instead of a Windows one. Now this is fixed and cursor files are generated with a .cur extension. Hopefully they will work OK.
gdk-pixbuf-loaders: I have added the cache file, and managed to get rid of the warning related to SVG files using this patch
icons: were did you get the additional icons that you have in your Adwaita subdir? Are they non-standard? I also notoced that you have lots of .png files in the images folder, that do not seem to be installed by RT...
debugging: the RT executable is now compiled in RelWithDebInfo mode, and the package contains a copy of gdb.exe from here.

Could you see if at least part of the problems are fixed for you as well with the new package?

gaaned92 commented 5 years ago

@aferrero2707

starts, runs 👍 Icons at top right 👍

cursors 👍 -gdk-pixbuf-loaders 👍 .
The adwaita subdirs are copied from MSYS2. The adwaita icon theme in MSYS2 is
```
mingw64/mingw-w64-x86_64-adwaita-icon-theme 3.28.0-1 [installé]
GNOME icon theme (mingw-w64)
```
as uploaded by pacman
.png files in the images subfolder: They are all installed by RT. The images subfolder in your buid seems exactly the same.
debugging: gdb 👍 . The relwithdebinfo build should be copied as rawtherapee-debug.exe in the package alogngside rawtherapee.exe.
- there are still some useless subdirs remining. Not important.
as a prerequisite to use the innosetup installer generator, did you succeded to generate windowsinnosetup.iss?

Thanks for the job

aferrero2707 commented 5 years ago

@gaaned92 thanks for checking so quickly!

Apart from the useless subdirs, do you still see any issue? I do not have access to a Win10 machine at the moment, so I am a little blind...

The relwithdebinfo build should be copied as rawtherapee-debug.exe in the package alogngside rawtherapee.exe

Is this really necessary? the rawtherapee-debug.exe command will run as fast as the stripped rawtherapee.exe, it just contains the debugging symbols (@heckflosse can probably confirm or correct my statement). Why bloating the package size for nothing?

The Innosetup stuff is my next step... no windowsinnosetup.iss generated for the moment ( I had to patch the CMake configuration to work around that).

Beep6581 commented 5 years ago

The package should contain either both release+debug, or one relwithdebinfo. I don't see sense in release+relwithdebinfo.

gaaned92 commented 5 years ago

@Beep6581

Default flags for release and relwithdebinfo usually are
```
CMAKE_CXX_FLAGS_RELEASE is -O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO is -O2 -g -DNDEBUG
```
Do you know if there is a significant performance gain between O2 an O3? I think so as I always was asked to build with O3 Or if you make a -O3 build with -g flag and symbol table, will the debug information be still useful?

As I am not dev, I have no conclusive answer on these questions.

@aferrero2707

I am on W10. I have no issue, no warning during start, no problem during processing. Behaviour is similar to that of my own builds. Nevertheless It has to be confirmed by thourough tests from other users. So for the moment I have no complaint
As you patch some RT files to permit travis build, do you intend to introduce those patches back in the main RT repository?

Beep6581 commented 5 years ago

http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html

@gaaned92 in my experience from Gentoo, most programs use -O2 and there was warnings against using -O3 claiming that programs weren't tested well using that and so you ran a risk of encountering bugs and artifacts.

The last time I benchmarked Release vs RelWithDebInfo was several years ago, probably before Ingo's work, and I measured total RT process time. Would be good to get some measurements using Release vs RelWithDebInfo and -O2 vs -O3 using the precise StopWatch and current RT code, e.g. total AMaZE time, total noise reduction time, total Fattal time. If someone could make a patch...

Beep6581 commented 5 years ago

My self-compiled build uses -O3, but I hadn't set that myself and I can't find where it was set - grep returned no hits for -O3 in CMake* nor in /etc/

According to a mailinglist @heckflosse just found in IRC, -O3 apparently is (or was in 2016) the GCC default: https://cmake.org/pipermail/cmake/2016-May/063379.html

As for Gentoo, from https://wiki.gentoo.org/wiki/GCC_optimization

-O3: the highest level of optimization possible. It enables optimizations that are expensive in terms of compile time and memory usage. Compiling with -O3 is not a guaranteed way to improve performance, and in fact, in many cases, can slow down a system due to larger binaries and increased memory usage. -O3 is also known to break several packages. Using -O3 is not recommended. However, it also enables -ftree-vectorize so that loops in the code get vectorized and will use AVX YMM registers.

RT explicitly sets -ftree-vectorize https://github.com/Beep6581/RawTherapee/blob/dev/CMakeLists.txt#L58

aferrero2707 commented 5 years ago

@Beep6581 @gaaned92 @heckflosse If the optimization options for Release and RelWithDebInfo targets differ, then I think it is wrong. After all, the idea is to debug exactly the same code as the production code, and it is known that certain bugs behave differently depending on the optimization options.

I would have a different scheme to propose:

for development branches, include the executables produced with RelWithDebInfo and Debug modes. This would allow to debug the "production-ready" code, but also check the bugs in the non-optimized code that is easier to interpret in some cases
for release packages, only include the optimized .exe without debugging symbols

What do you think?

gaaned92 commented 5 years ago

I have no definite position though I think it's an interesting proposal. What I notice is that a build without symbol table is about 9GB wheras a build with symbol table is 187 MB. Thus the solution above should increase size of installation by about 180MB for a grand total of about 500MB. I don't know if it is a problem now.

@aferrero2707 look at your gtk3 and gtkmm3 dlls. They seem to include a symbol table. In photoflow package, they are much smaller.

aferrero2707 commented 5 years ago

Dear developers, I have prepared a new version of the AppImage packages that fix some issues with the LensFun database loading and with the invocation of external editors (like GIMP).

It would be great if somebody could test those two features with the latest packages, and let me know if there are still some problems...

Thanks!

Beep6581 commented 5 years ago

Tested in Sabayon: https://github.com/Beep6581/RawTherapee/releases/download/nightly/RawTherapee-dev-5.4-792-g3643e15.AppImage

Lensfun works fine, and "Edit current image in external editor" starts my GIMP-2.10.4 correctly.

Beep6581 commented 5 years ago

Hi @gaaned92 , you mentioned that some icons no longer need to be copied and other build process changes - are our Windows build instructions up to date and valid? http://rawpedia.rawtherapee.com/Windows

aferrero2707 commented 5 years ago

@gaaned92 I still need to clean up and strip the unneeded libraries... meanwhile, I have started to look into the InnoSetup stuff. I already managed to install it and run it under wine. It seems it will be possible to include it in the automated packaging that is run in the Docker container.

Could you give me some hints on how to invoke ISCC.exe and create an installer for RT?

Thanks!

heckflosse commented 5 years ago

@aferrero2707

@gaaned92 is afk for the next about 3 to 4 weeks

aferrero2707 commented 5 years ago

Right, I forgot that! Then I'll try to figure out myself how to use innosetup...

aferrero2707 commented 5 years ago

@heckflosse I have managed to include the InnoSetup part in my automated windows build. The ISCC compiler is executed using wine, which seems to work perfectly fine.

Would you have the possibility to test the installer and see if you spot some issues? It is available from here: https://github.com/aferrero2707/rt-win64/releases/download/continuous/RawTherapee_dev-5.4-854-ge99bcff8_WinVista_64.exe

Thanks!

heckflosse commented 5 years ago

@aferrero2707 I didn't test thoroughly but at least it installed fine here.

aferrero2707 commented 5 years ago

@heckflosse thanks for checking, that's already a good indication that the innosetup compiler works correctly under wine.

SilvioGrosso commented 5 years ago

Hello @aferrero2707

Just tested this new build on Windows 10 - 64 bit CPU I7; RAM 8gb. Tested on my NEF files (from my Nikon D700).

Everything is fine. I have tested this new (RawTherapee_dev-5.4-854-ge99bcff8_WinVista_64).exe for around 1 hour. No crash whatsoever and it looks as fast as the previous ones.

aferrero2707 commented 5 years ago

@SilvioGrosso @heckflosse

Thanks for checking. Meanwhile I have cleaned up the package contents and stripped the debugging symbols from all the accessory libraries (but not the rawtherapee executable), as suggested by @gaaned92. I have also renamed the packages to follow the usual RT conventions. Here are the direct links to the new packages: https://github.com/aferrero2707/rt-win64/releases/download/continuous/RawTherapee_dev-5.4-854-ge99bcff8.zip https://github.com/aferrero2707/rt-win64/releases/download/continuous/RawTherapee_dev-5.4-854-ge99bcff8_WinVista_64.exe

It would be great if you could test them again, and eventually give at some point green light for publishing the packages in the "nightly builds" release page together with the AppImages.

The installer and the .zip versions are actually identical in their contents, so they should work really in the same way.

Testing with wine, I noticed at least one icon missing (see screenshot). I do not know if it is also missing when running natively under Windows.

SilvioGrosso commented 5 years ago

Hello @aferrero2707

As suggested, I have just tested these new two builds (ZIP and .exe) on Windows 10 - 64 bit CPU I7; RAM 8gb. Tested on my NEF files, from my Nikon D700.

Both builds are missing many icons, much more than the previous one. For example, take a look at the right upper side (these 3 icons are extremely prone to being missing in all the builds I have tested so far, both on Linux and Windows). Aside from that, I didn't encounter any problem, in essence, any crash.

See this screenshot:

BTW, I have just signed in to become a Patron of your project: given your efforts so far, you really deserve it ! :-)

aferrero2707 commented 5 years ago

@SilvioGrosso I probably removed too many icons from the packages... so I put them back for now, until I'll have a chance to check the details with @gaaned92.

Could you please check if the latest packages do have all the required icons?

P.S: I really appreciate your support, Thanks!

aferrero2707 commented 5 years ago

I have introduced some improvements in the desktop integration of the RT AppImage. I basically worked on two aspects:

integration with the applications menu and the "Open with..." contextual menu of the file managers. For this to work, make sure you have the desktop-file-install installed command in your system. When launching the AppImage for the first time, a dialog like the attached one should pop up, asking wether you want to enable the desktop integration. After the integration has been successfully completed, an entry called RawTherapee (AppImage) should appear in your applications menu, under the Graphics sub-menu
Automated download of updates, when available. The AppImage should check for updates in the nightly builds release, and a dialog should pop up proposing the installation of the latest available package if it is more recent than the AppImage being executed. If the download is accepted, the new AppImage is saved in the $HOME/.local/appimages/ and is immediately executed. It will also ask the user if he/she wants to update the desktop integration. curl needs to be installed for this to work.

There are few aspects that still need improvements:

the updates are searched each time the AppImage is executed. This should probably be limited to once a day
although the GitHub releases should be quite safe, there is no check yet for the integrity and validity of the downloaded AppImage. The format allows to embed a gpg signature in the package itself, but I still need to dig into the details...
previously downloaded packages are not automatically removed.
the download script should be adapted to use wget if curl is not available.

Could you please test the experimental package, and let me know if the new features works for you? Suggestions are obviously welcome! The package is available at this link: https://filebin.net/hgqz4r8isdls3ze9

The RT version is intentionally a bit outdated, so that updates are found for sure.

Beep6581 / RawTherapee

Automated AppImage and Windows builds with Travis CI #4404