Wrong paths in post-installation deployment scripts

sbjartmar commented 11 months ago

Hi all, I'm working at DMSC (ESS) in Copenhagen setting up a container to deploy Muhrec. I followed the instructions on how to install the imaging suite, and cmake installs it with no problems. It seems that the scripts deploy_pythonbindings_ubuntu.sh and deploymuhrec_ubuntu_withQT.sh have some issues.

deploymuhrec_ubuntu_withQT.sh has hardcoded values for extracting the version from the QT path, so if one doesn't have a tilde in the front, it doesn't set the QtV and QtVmain correctly, e.g. to .2.4 not 6.2.4.
In deploymuhrec_ubuntu_withQT.sh, the $WORKSPACE variable, which points towards ~/git based on the installation script, here seems to be assumed to point at the repo path ~/git/imagingsuite.
The installation doesn't create the$WORKSPACE/scripts/python/amglib/*.py mentioned in deploy_pythonbindings_ubuntu.sh. Does that have to be installed separately?

I've modified the scripts to point towards the correct libraries, but amglib seems to be missing. Am I doing something wrong?

Here's my installation script:

apt-get update -y && apt-get upgrade -y
apt-get install -y wget build-essential cmake git ninja-build \
                   libcfitsio-dev libtiff-dev libxml2-dev libnexus1 libnexus-dev \
                   libfftw3-dev libarmadillo10 libarmadillo-dev pybind11-dev \
                   libx11-dev xorg-dev libglu1-mesa-dev freeglut3-dev \
                   libglew2.2 libglew-dev libglu1-mesa libglu1-mesa-dev \
                   libgl1-mesa-glx libgl1-mesa-dev libglfw3-dev libglfw3 \
                   libxkbcommon-dev libicu-dev libtiff5-dev libfftw3-dev libblas-dev \
                   libhdf5-dev python3 python3-pip libdbus-1-3 libpulse-mainloop-glib0

# Install Miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /tmp/miniconda.sh
bash /tmp/miniconda.sh -b -p /opt/conda
rm /tmp/miniconda.sh

# Set Conda paths for the rest of the script
export PATH="/opt/conda/bin:$PATH"
conda init

# Install Conda dependencies
conda install -y cmake ninja pybind11

# Extract and install Qt 6.2.4  
export QTPATH=/qt
pip3 install aqtinstall
aqt install-qt --outputdir $QTPATH linux desktop 6.2.4 gcc_64 -m all

for tool in \
    tools_telemetry_gui \
    tools_telemetry_eval_gui \
    tools_telemetry \
    tools_qtdesignstudio \
    tools_qtcreator_gui \
    tools_qtcreator \
    tools_qt3dstudio \
    tools_opensslv3_src \
    tools_ninja \
    tools_maintenance_update_reminder \
    tools_maintenance \
    tools_ifw \
    tools_generic \
    tools_conan \
    tools_cmake \
    sdktool
do
    aqt install-tool --outputdir $QTPATH linux desktop $tool
done

# Clone the repository and set workspace
mkdir -p ~/git
git clone https://github.com/neutronimaging/imagingsuite.git ~/git/imagingsuite

# Set environment variables
export QTBINPATH=$QTPATH/6.2.4/gcc_64/bin
export WORKSPACE=~/git
export CONDA_PREFIX=/opt/conda

# Create build directory and build project
cd $WORKSPACE
mkdir -p build install
cd build
/qt/Tools/CMake/bin/cmake ../imagingsuite \
    -DCMAKE_INSTALL_PREFIX=../install \
    -DPYTHON_LIBRARY=$CONDA_PREFIX/lib/libpython3.11.so \
    -DPYTHON_INCLUDE_DIR=$CONDA_PREFIX/include/python3.11 \
    -DPYTHON_EXECUTABLE=$CONDA_PREFIX/bin/python \
    -DCMAKE_PREFIX_PATH=$QTPATH/6.2.4/gcc_64 \
    -G="Ninja" 

cmake --build . --target install
cd ..
# Deployment
chmod +x $WORKSPACE/deploymuhrec_ubuntu_withQT.sh
chmod +x $WORKSPACE/deploy_pythonbindings_ubuntu.sh
$WORKSPACE/deploymuhrec_ubuntu_withQT.sh
$WORKSPACE/deploy_pythonbindings_ubuntu.sh

Here's the custom deploymuhrec_ubuntu_withQT.sh:

#!/bin/bash
CPCMD="cp "
REPOSPATH=$WORKSPACE

QT_PATH="$QTPATH"
QtV="6.2.4"
COMPILER=gcc_64

DEST="$REPOSPATH/deploy/muhrec"

mkdir -p $DEST/bin/platforms $DEST/Frameworks $DEST/resources

pushd .
cd $DEST/Frameworks
rm -f *.1.0.0.dylib

libs=( "libQtAddons.so" "libQtModuleConfigure.so" "libImagingAlgorithms.so" "libkipl.so" \
            "libModuleConfig.so" "libReaderConfig.so" "libReaderGUI.so" "libReconFramework.so" \
            "libReconAlgorithms.so" "libStdBackProjectors.so" "libStdPreprocModules.so" "libStdPreprocModulesGUI.so" \
            "libInspectorModules.so" "libInspectorModulesGUI.so" "libFDKBackProjectors.so" "libQtImaging.so" )

libsQT=( "libQt6Core.so.$QtV" "libQt6Gui.so.$QtV" "libQt6Widgets.so.$QtV" \
            "libQt6DBus.so.$QtV" "libQt6PrintSupport.so.$QtV" "libQt6XcbQpa.so.$QtV" \
            "libQt6Charts.so.$QtV" "libicui18n.so.56.1" "libicudata.so.56.1" "libicuuc.so.56.1" )

echo "Copying regular libs"
for lib in "${libs[@]}"; do
    cp "$REPOSPATH/install/lib/$lib" .
done

echo "Copying QT libs"
for lib in "${libsQT[@]}"; do
    cp "$QT_PATH/$QtV/$COMPILER/lib/$lib" .
done

echo "for loop 1"
for f in `ls *.so.$QtV`; do
    bn=`basename $f .so.$QtV`
    ln -s $f $bn.so.$QtVmain
        ln -s $bn.so.$QtVmain $bn.so
    ln -s $bn.so $bn.so
done

echo "for loop 2"
rm -f *.so6
for f in `ls *.so.6.2.4`; do
    bn=`basename $f .so.6.2.4`
    ln -s $bn.so.6.2.4 $bn.so.6
    ln -s $bn.so.6 $bn.so
done
echo "weird cpmcd"

`$CPCMD $QT_PATH/$QtV/$COMPILER/plugins/platforms/libqxcb.so $DEST/bin/platforms`
`$CPCMD $REPOSPATH/install/applications/MuhRec $DEST/bin`
`$CPCMD $REPOSPATH/imagingsuite/applications/muhrec/scripts/muhrec $DEST`
chmod +x $DEST/muhrec

cp $REPOSPATH/imagingsuite/applications/muhrec/Resources/*.xml $DEST/resources

fname=MuhRec-Ubuntu_`uname -s`_`uname -m`_build-`date +%Y%m%d`.tar.bz2
echo $fname 

cd $DEST
cd ..
tar -jcvhf $fname 'muhrec'

popd

Cheers, Siggi

anderskaestner commented 11 months ago

Hi Siggi,

The ubuntu deployment scripts haven't been updated in a while. This is something, I plan to work on in the near future. Here, I plan to use https://github.com/probonopd/linuxdeployqt for the Qt part, but I'm not there yet.

The missing scripts $WORKSPACE/scripts/python/amglib/*.py are in a separate repository. https://github.com/neutronimaging/scripts

Cheers,

Anders

sbjartmar commented 11 months ago

Hi Anders, thanks for the swift reply. Managed to get it to work after cloning the scripts repo. Thanks for the help. Fine by me if you want to close this.

Siggi

anderskaestner commented 11 months ago

Ok

neutronimaging / imagingsuite

Wrong paths in post-installation deployment scripts #619