search

CBICA / CaPTk

Cancer Imaging Phenomics Toolkit (CaPTk) is a software platform to perform image analysis and predictive modeling tasks. Documentation: https://cbica.github.io/CaPTk
https://www.cbica.upenn.edu/captk
Other
180 stars 64 forks source link

Linux package needs to be compiled on Ubuntu 14.04 or Centos 7 #505

Closed MarkBergman-cbica closed 4 years ago

MarkBergman-cbica commented 5 years ago 
wget -nd ftp://www.nitrc.org/home/groups/captk/downloads/CaPTk_1.7.0.Beta_Installer.bin
--2019-05-14 10:40:28--  ftp://www.nitrc.org/home/groups/captk/downloads/CaPTk_1.7.0.Beta_Installer.bin
           => ‘CaPTk_1.7.0.Beta_Installer.bin’
Resolving www.nitrc.org (www.nitrc.org)... 132.239.16.23
Connecting to www.nitrc.org (www.nitrc.org)|132.239.16.23|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /home/groups/captk/downloads ... done.
==> SIZE CaPTk_1.7.0.Beta_Installer.bin ... 1072223624
==> PASV ... done.    ==> RETR CaPTk_1.7.0.Beta_Installer.bin ... done.
Length: 1072223624 (1023M) (unauthoritative)

CaPTk_1.7.0.Beta_Installer.bin      100%[=================================================================>]   1023M  12.2MB/s    in 2m 51s  

2019-05-14 10:43:20 (5.98 MB/s) - ‘CaPTk_1.7.0.Beta_Installer.bin’ saved [1072223624]

[bergman@fff tmp]$ sh ./Ca*bin
Verifying archive integrity...  100%   All good.
Uncompressing CaPTk Linux Installer  100%  
By installing the Cancer and Phenomics Toolkit (CaPTk), the user agrees to the following licenses, which pertain to the code and its different
 constituents:

1. SBIA Software License - https://www.med.upenn.edu/sbia/software-agreement.html
2. SBIA Non-Commercial Software License (applies to PHI Estimator) - https://www.med.upenn.edu/sbia/software-agreement-non-commercial.html
3. Insight Toolkit License - https://cmake.org/Wiki/ITK/License_Information
4. Visualization Toolkit License - http://www.vtk.org/licensing/
5. Qt Open Source License - https://doc.qt.io/qt-5.10/qtcore-index.html#licenses-and-attributions; our copy can be found at https://github.com
/CBICA/qt
6. CC Attribution-ShareAlike for SRI24 data - https://creativecommons.org/licenses/by-sa/3.0/us/legalcode 
7. MRIcroGL License - BSD
8. YAML-CPP - MIT License and can be found at https://opensource.org/licenses/MIT
9. Eigen - MPL2 License and can be found at https://www.mozilla.org/en-US/MPL/2.0/

I accept the terms of the licenses above [Y/N] y
~/CaPTk exists. Proceeding...
Creating ~/CaPTk/1.7.0.Beta/...
Checking disk space...
Checking for FUSE...
fusermount version: 2.9.9
Installing... \
Installing CWL
Creating symbolic link 'captk'
Done.
To run CaPTk, enter the command "~/CaPTk/1.7.0.Beta/captk" in your console.
[bergman@fff tmp]$ ~/CaPTk/1.7.0.Beta/captk
/tmp/.mount_captkWmS9AW/usr/bin/CaPTk: /tmp/.mount_captkWmS9AW/usr/lib/libQt5Core.so.5: version `Qt_5.12' not found (required by /tmp/.mount_captkWmS9AW/usr/bin/CaPTk)
/tmp/.mount_captkWmS9AW/usr/bin/CaPTk: /tmp/.mount_captkWmS9AW/usr/lib/libQt5Core.so.5: version `Qt_5.12' not found (required by /tmp/.mount_captkWmS9AW/usr/lib/libQt5WebView.so.5)
/tmp/.mount_captkWmS9AW/usr/bin/CaPTk: /tmp/.mount_captkWmS9AW/usr/lib/libQt5Core.so.5: version `Qt_5.12' not found (required by /tmp/.mount_captkWmS9AW/usr/lib/libQt5WebEngine.so.5)
/tmp/.mount_captkWmS9AW/usr/bin/CaPTk: /tmp/.mount_captkWmS9AW/usr/lib/libQt5Core.so.5: version `Qt_5.12' not found (required by /tmp/.mount_captkWmS9AW/usr/lib/libQt5WebEngineWidgets.so.5)
sarthakpati commented 5 years ago

Was able to replicate this on the cluster.

@jminock: could you try installing & running CaPTk using this link?

sarthakpati commented 5 years ago

After extracting the AppImage, I am getting the following error on the cluster:

patis@/cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin>$ ./CaPTk
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by ./CaPTk)
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by /cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin/../lib/libQt5WebView.so.5)
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by /cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin/../lib/libQt5WebEngine.so.5)
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by /cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin/../lib/libQt5WebEngineWidgets.so.5)
MarkBergman-cbica commented 5 years ago

Downloaded the file linked from Azure:

[bergmama@cubic-login2 CBICA.CaPTk_installer_Linux]$ ~/CaPTk/1.7.0.RC/captk
/scratch/bergmama/.mount_captkQWMbDO/usr/bin/CaPTk: /cbica/software/external/gcc/centos7/4.9.2/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /scratch/bergmama/.mount_captkQWMbDO/usr/bin/CaPTk)
/scratch/bergmama/.mount_captkQWMbDO/usr/bin/CaPTk: /usr/lib64/libgpg-error.so.0: no version information available (required by /scratch/bergmama/.mount_captkQWMbDO/usr/lib/libgcrypt.so.20)
[bergmama@cubic-login2 CBICA.CaPTk_installer_Linux]$
sarthakpati commented 5 years ago

/scratch/bergmama/.mount_captkQWMbDO/usr/bin/CaPTk: /cbica/software/external/gcc/centos7/4.9.2/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /scratch/bergmama/.mount_captkQWMbDO/usr/bin/CaPTk)

This package will not work on the cluster since it is getting built on Ubuntu 16.04 so this error is expected.

/scratch/bergmama/.mount_captkQWMbDO/usr/bin/CaPTk: /usr/lib64/libgpg-error.so.0: no version information available (required by /scratch/bergmama/.mount_captkQWMbDO/usr/lib/libgcrypt.so.20)

I haven't seen this before.

sarthakpati commented 5 years ago

After extracting the AppImage, I am getting the following error on the cluster:

patis@/cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin>$ ./CaPTk
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by ./CaPTk)
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by /cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin/../lib/libQt5WebView.so.5)
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by /cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin/../lib/libQt5WebEngine.so.5)
./CaPTk: /usr/lib64/libQt5Core.so.5: version `Qt_5.12' not found (required by /cbica/home/patis/CaPTk/1.7.0.Beta/squashfs-root/usr/bin/../lib/libQt5WebEngineWidgets.so.5)

Upon seeing the output generated while creating the installer on the cluster, I noticed the following lines:

2019-05-14 12:06:52 (86.0 MB/s) - ‘linuxdeployqt-6-x86_64.AppImage’ saved [14720712/14720712] [SNIP!] iconToBeUsed: "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/share/icons/hicolor/64x64/CaPTk.png" Copied "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/share/icons/hicolor/64x64/CaPTk.png" to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/CaPTk.png" ERROR: file copy failed from "/usr/lib64/qt5/libexec/QtWebEngineProcess" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/libexec/QtWebEngineProcess" ERROR: file copy failed from "/usr/share/qt5/resources/qtwebengine_resources.pak" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/resources/qtwebengine_resources.pak" ERROR: file copy failed from "/usr/share/qt5/resources/qtwebengine_devtools_resources.pak" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/resources/qtwebengine_devtools_resources.pak" ERROR: file copy failed from "/usr/share/qt5/resources/qtwebengine_resources_100p.pak" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/resources/qtwebengine_resources_100p.pak" ERROR: file copy failed from "/usr/share/qt5/resources/qtwebengine_resources_200p.pak" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/resources/qtwebengine_resources_200p.pak" ERROR: file copy failed from "/usr/share/qt5/resources/icudtl.dat" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/resources/icudtl.dat" ERROR: findDependencyInfo: "ldd: /usr/lib64/qt5/plugins/iconengines/libqsvgicon.so: No such file or directory\n" ERROR: getBinaryRPaths: "/cbica/software/external/gcc/centos7/4.9.2/bin/objdump: '/usr/lib64/qt5/plugins/iconengines/libqsvgicon.so': No such file\n" ERROR: file copy failed from "/usr/lib64/qt5/plugins/iconengines/libqsvgicon.so" ERROR: to "/cbica/home/patis/projects/captk_myFork/bin/install/appdir/usr/bin/..//plugins/iconengines/libqsvgicon.so" ERROR: Could not find any translations in "/usr/share/qt5/translations" (developer build?)

Which would explain the Qt issues.

However, what is curious is why the package generated from Ubuntu 16.04 is totally fine (on machines with compatible GLIBC versions) even when the host machine doesn't have Qt5 installed.

sarthakpati commented 5 years ago

The package generated by Azure works perfectly fine on my WSL (Ubuntu 16.04); both CLI and GUI. All required Qt libraries are present.

sarthakpati commented 5 years ago

@PhucNgo1711: There is a project on GitHub called LightGMB [ref] and it uses a combination of an old Trusty Docker (we can use this, which is building Qt) to maintain GLIBC compatibility. Their Azure script gives the mechanism to do this. I believe if we can combine our build script with theirs, it should give us what we want? Of course, we want to keep this independent to ensure all this works as expected.

sarthakpati commented 5 years ago

Compiling on Ubuntu 14.04 using newer Qt is a no-go unless we recompile Qt from source [ref]. Let's see if compiling on the cluster yields better results.

sarthakpati commented 5 years ago

Recompiled the package on CentOS 7 using the installed Qt 5.11 version. I still get the same issue, even though I can clearly see that the shared libraries the executable is trying to link to is present where is should be.

One possible solution we need to explore now is to compile the CaPTk installer without calling LinuxDeployQt and let the user provide the required Qt libraries.

sarthakpati commented 5 years ago

Packaged CaPTk, installed and extracted from the AppImage to see the same issue (Qt version mismatch). Upon checking the QtCore library getting packaged, I came upon the following:

(base) patis@/cbica/home/patis/Downloads/squashfs-root/usr/bin>$ strings ../lib/libQt5Core.so.5 | grep -i version
/etc/debian_version
Displays version information.
version
Invalid XML version string.
Unsupported XML version.
<?xml version="
This is the QtCore library version Qt 5.9.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 4.8.5 20150623 (Red Hat 4.8.5-28))
[SNIP!]

Which basically means that LinuxDeployQt is putting in the system version of Qt, rather than the Qt that CaPTk gets built against.

Possible solutions:

  1. Build CaPTk (dependencies + final executables) using the system version of Qt on CentOS7 (which is 5.9.2): unsure if this will work because of some Qt dependencies we have
  2. Specifically ask LinuxDeployQt to look at the QMake we want [ref]
  3. Create a CentOS7 VM, ensure Qt is not installed in the system and then use the Qt binaries we ship
sarthakpati commented 5 years ago

The commit https://github.com/sarthakpati/CaPTk/commit/1277fde3c323acf9192ad4ee67cdf6c8b9c0b9d3 fixes this. Closing.

sarthakpati commented 5 years ago

LinuxDeployQt does not support CentOS.

Reference: https://github.com/probonopd/linuxdeployqt/issues/388#issuecomment-523419799

PhucNgo1711 commented 5 years ago

Tested the latest development build on the cluster. This issue has emerged again. 

/scratch/ngoph/.mount_captk4xBKj3/usr/bin/CaPTk: /cbica/software/external/gcc/centos7/4.9.2/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /scratch/ngoph/.mount_captk4xBKj3/usr/bin/CaPTk)
/scratch/ngoph/.mount_captk4xBKj3/usr/bin/CaPTk: /usr/lib64/libgpg-error.so.0: no version information available (required by /scratch/ngoph/.mount_captk4xBKj3/usr/lib/libgcrypt.so.20)
sarthakpati commented 5 years ago

What is the executable name? The "CentOS7" build under the dev downloads section should work on the cluster since it was compiled there.