Install script and source building

[pfranchini]

Would be possible to create an install scripts taking care of the whole installation, treating all the core libraries as third parties, doing the download and build?

Very low priority of course, but can be really useful.

Thanks.

[fmauger]

Hi Paolo,

I already have some stuff for that for my own usage : devel environment, test new versions of Boost, G4, ROOT... with fast build/instal (not so fast...). It is in https://github.com/BxCppDev/BxInstallers where I've written a bunch of individual installers for some big software blocks needed for Bayeux. It works reasonably well on Ubuntu 18.04 but there are still some bugs and it misses a documentation and a real life example.

However, it mimics the behavior of systems like brew which I found to be a nightmare after a few years. But I'm not sure it makes sense to provide such a tool for our community. There are new software on the market like Spack to solve in principle this painful tasks : build, install, manage deps with various releases... I think we should have a look on it in a near future to provide a unique and (I hope) working tool to ease the job.

Nevertheless I will have a look on BxInstallers to see if it can help you in the meanwhile. Cheers

[pfranchini]

Hi Francois,

thanks for that. The pain right now for me is to install everything in a cluster machine where I do not have sudo access and where everything coming by default seems to be very outdated.

Speaking of which have you got any idea of to solve this

CMake Error at /vols/build/snemo/CMake/Modules/FindPackageHandleStandardArgs.cmake:137 (message): Could NOT find GSL: Found unsuitable version "1.15", but required is at least "2.1" (found /vols/build/snemo/gsl-2.4)

Have installed it on a separate directory but struggling to convince cmake to find it,

cheers,

Paolo

On 19/09/2019 17:39, François Mauger wrote:

Hi Paolo,

I already have some stuff for that for my own usage : devel environment, test new versions of Boost, G4, ROOT... with fast build/instal (not so fast...). It is in https://github.com/BxCppDev/BxInstallers where I've written a bunch of individual installers for some big software blocks needed for Bayeux. It works reasonably well on Ubuntu 18.04 but there are still some bugs and it misses a documentation and a real life example.

However, it mimics the behavior of systems like brew which I found to be a nightmare after a few years. But I'm not sure it makes sense to provide such a tool for our community. There are new software on the market like Spack to solve in principle this painful tasks : build, install, manage deps with various releases... I think we should have a look on it in a near future to provide a unique and (I hope) working tool to ease the job.

Nevertheless I will have a look on BxInstallers to see if it can help you in the meanwhile. Cheers

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BxCppDev/Bayeux/issues/54?email_source=notifications&email_token=AIU7CTFWXCKWJPBUES7THQ3QKOTMPA5CNFSM4IYM6UY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7EC7LI#issuecomment-533213101, or mute the thread https://github.com/notifications/unsubscribe-auth/AIU7CTEBS5TZB5H2ZX4IN3DQKOTMPANCNFSM4IYM6UYQ.

[emchauve]

In case it can help : the solution I am deploying usually is to start an installation of falaise with SuperNEMO-DBD/brew and aborting it when I reach the step of Bayeux and Falaise. Then I can install them separately in different directories and handle different blessed versions and others under development :

[...]/software/Homebrew/...

[...]/software/Bayeux/3.4.1/... [...]/software/Bayeux/develop/...

[...]/software/Falaise/4.0.0/... [...]/software/Falaise/develop/...

to setup environment, for exemple in Falaise/4.0.0 directory, there is a script thisFalaise.sh like :

#!/bin/bash

source [...]/Bayeux/3.4.1/thisBayeux.sh

export FL_BASE_PATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
echo "falaise:  $FL_BASE_PATH"

export  FL_SOURCE_PATH=$FL_BASE_PATH/source
export   FL_BUILD_PATH=$FL_BASE_PATH/build
export FL_INSTALL_PATH=$FL_BASE_PATH/install

export PATH=$FL_INSTALL_PATH/bin:$PATH
export LD_LIBRARY_PATH=$FL_INSTALL_PATH/lib64:$LD_LIBRARY_PATH

which load the associated thisBayeux.sh :

#!/bin/bash

source[...]/Homebrew/thisHomebrew.sh

export BX_BASE_PATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
echo "bayeux:  $BX_BASE_PATH"

export  BX_SOURCE_PATH=$BX_BASE_PATH/source
export   BX_BUILD_PATH=$BX_BASE_PATH/build
export BX_INSTALL_PATH=$BX_BASE_PATH/install

export PATH=$BX_INSTALL_PATH/bin:$PATH
export LD_LIBRARY_PATH=$BX_INSTALL_PATH/lib64:$LD_LIBRARY_PATH

which load the base thisHomebrew.sh :

#!/bin/bash

export HOMEBREW_REPOSITORY=[...]/Homebrew
export HOMEBREW_PREFIX=[...]/Homebrew

export MANPATH=[...]/Homebrew/share/man

export PATH=$HOMEBREW_PREFIX/bin:$HOMEBREW_PREFIX/sbin:$PATH
export INFOPATH=$HOMEBREW_PREFIX/share/info
export LD_LIBRARY_PATH=[...]/Homebrew/lib:$LD_LIBRARY_PATH

export HOMEBREW_CELLAR=$HOMEBREW_PREFIX/Cellar

cmake is working fine for me like that

[pfranchini]

Hi Francois, thanks for that. The pain right now for me is to install everything in a cluster machine where I do not have sudo access and where everything coming by default seems to be very outdated. Speaking of which have you got any idea of to solve this CMake Error at /vols/build/snemo/CMake/Modules/FindPackageHandleStandardArgs.cmake:137 (message): Could NOT find GSL: Found unsuitable version "1.15", but required is at least "2.1" (found /vols/build/snemo/gsl-2.4) Have installed it on a separate directory but struggling to convince cmake to find it, cheers, Paolo

This was solved with -DGSL_ROOT_DIR=/vols/build/snemo/gsl-2.4/

[pfranchini]

Still in the process of scripting a building from sources...

First had to remove in Bayeux the flag -Wshadow from cmake/LPCCMakeSettings.cmake because was incompatible with geant4.9.6.p04. Next got this

[pfranchi@lx02 Bayeux.build]$ make
[  5%] Built target Bayeux_mctools_geant4
[  5%] Linking CXX shared library ../BuildProducts/lib64/libBayeux.so
c++: error: camp::camp-NOTFOUND: No such file or directory
make[2]: *** [BuildProducts/lib64/libBayeux.so.3.4.1] Error 1
make[1]: *** [source/CMakeFiles/Bayeux.dir/all] Error 2
make: *** [all] Error 2

while cmake had found CAMP without any problem

-- Found CAMP at CAMP_DIR      = '/vols/build/snemo/camp'
-- CAMP_INCLUDE_DIRS     = '/vols/build/snemo/camp/include'

Is full of camp::camp-NOTFOUND all over the places in the source/CMakeFiles c++ instructions Any idea? Thanks!

[fmauger]

In principle you should not use the LD_LIBRARY_PATH in your setup scripts. Bayeux et Falaise software use the RPATH mechanism to indentify unambiguously the shared libraries to be linked.

About CAMP, what version is installed in your /vols/build/snemo/camp dir. It seems the Camp 0.8 lacks some CAMPConfig.cmake file to automate CMake's find_package. I now use CAMP 0.8.2 from https://github.com/fw4spl-org/camp. cheers frc

[pfranchini]

Currently on https://github.com/fw4spl-org/camp is 0.8.3. 0.8.3 gives this

/vols/build/snemo/camp/include/camp/detail/rawtype.hpp:100:38: error: ‘enable_if’ in namespace ‘boost’ does not name a type
 struct RawType<T<U>, typename boost::enable_if<IsSmartPointer<T<U>, U> >::type>
                                      ^
compilation terminated due to -Wfatal-errors.

So, I have cloned 0.8.2 and I am still getting the same problem with camp::camp-NOTFOUND and this time used sed to point at the built object. Moving further I get this in the make process of Bayeux

[ 95%] Linking CXX executable ../BuildProducts/bin/bxg4_seeds
/usr/bin/ld: _ZN5CLHEP9RandGauss6set_stE: TLS definition in /vols/build/snemo/CLHEP.build/lib/libCLHEP-2.4.1.2.so section .tbss mismatches non-TLS definition in /vols/build/snemo/geant4.9.6.p04.build/outputs/library/Linux-g++/libG4clhep.so section .bss
/vols/build/snemo/geant4.9.6.p04.build/outputs/library/Linux-g++/libG4clhep.so: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [BuildProducts/bin/bxg4_seeds-3.4.1] Error 1
make[1]: *** [source/CMakeFiles/bxg4_seeds.dir/all] Error 2
make: *** [all] Error 2

[fmauger]

It seems your G4 has its own CLHEP built and likely conflicts with the standalone CLHEP built, maybe because of LD_LIBRARY_PATH that should not be used.

[pfranchini]

I have now downgraded to CLHEP-2.4.1.0 and rebuilding both CLHEP and G4. My LD_LIBRARY_PATH is empty. Same error.

[drbenmorgan]

Note that SuperNEMO's brew system applies several patches to Geant4, CAMP etc to get the required versions to build, as well as apply the needed build settings.

@emchauve's approach is the easiest and best at present. You can also do a full install up to Bayeux/Falaise and then do

$ brew unlink bayeux
... if Falaise was installed
$ brew unlink falaise

The unlink just guarantees a clean build environment without Bayeux headers/libs appearing in include/link paths etc.

[pfranchini]

I leave here my dirty recipe done on:

CentOS Linux release 7.7.1908 (Core)
cmake version 3.15.20190802-ga1e3262
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
ROOT Version: 6.18/04

From boost to Falaise:

wget https://dl.bintray.com/boostorg/release/1.63.0/source/boost_1_63_0.tar.gz
./bootstrap.sh --prefix=/vols/build/snemo/boost_1_63_0
./b2

git clone https://github.com/fw4spl-org/camp --branch=0.8.2
mkdir camp.build; cd camp.build
/vols/build/snemo/CMake/bin/cmake -DCMAKE_INSTALL_PREFIX=/vols/build/snemo/camp.build/ -DPACKAGE_PREFIX_DIR=/vols/build/snemo/camp.build ../camp
make
make install
edit CAMPConfig.cmake for set_and_check(CAMP_INCLUDE_DIR "/vols/build/snemo/camp/include")
cp /vols/build/snemo/camp.build/lib/camp/cmake/CAMPTargets.cmake .

git clone https://gitlab.cern.ch/CLHEP/CLHEP.git --branch=CLHEP_2_1_3_1
mkdir CLHEP.build; cd CLHEP.build
/vols/build/snemo/CMake/bin/cmake -DCMAKE_INSTALL_PREFIX=/vols/build/snemo/CLHEP.build ../CLHEP/
make -j2
make install

wget http://mirrors.ibiblio.org/gnu/ftp/gnu/gsl/gsl-2.4.tar.gz
cd gsl-2.4
./configure
make
make install prefix=/vols/build/snemo/gsl-2.4

wget cern.ch/geant4-data/releases/geant4.9.6.p04.tar.gz
mkdir data; cd data
wget http://cern.ch/geant4-data/datasets/G4EMLOW.6.32_permissions.tar.gz
mkdir geant4.9.6.p04.build; cd geant4.9.6.p04.build
/vols/build/snemo/CMake/bin/cmake -DGEANT4_USE_GDML=ON -DCLHEP_ROOT_DIR=/vols/build/snemo/CLHEP.build -DGEANT4_USE_SYSTEM_CLHEP=ON  ../geant4.9.6.p04
make -j2

cd Bayeux.build/
edited /vols/build/snemo/Bayeux/cmake/LPCCMakeSettings.cmake to remove the flag -Wshadow
/vols/build/snemo/CMake/bin/cmake -DCMAKE_INSTALL_PREFIX=/vols/build/snemo/Bayeux.build -DBOOST_ROOT=/vols/build/snemo/boost_1_63_0 -DGSL_ROOT_DIR=/vols/build/snemo/gsl-2.4/ -DCMAKE_PREFIX_PATH="/vols/build/snemo/camp.build;/vols/build/snemo/CLHEP.build/;/vols/build/snemo/geant4.9.6.p04.build/" ../Bayeux
find ./ -type f -exec sed -i +s+camp::camp-NOTFOUND+/vols/build/snemo/camp.build/libcamp.so+ {} \;
make
make install

git clone https://github.com/supernemo-dbd/Falaise.git --branch=v4.0.1
mkdir Falaise.build; cd Falaise.build
/vols/build/snemo/CMake/bin/cmake -DCMAKE_INSTALL_PREFIX=/vols/build/snemo/Falaise.build -DGSL_ROOT_DIR=/vols/build/snemo/gsl-2.4/ -DCMAKE_PREFIX_PATH="/vols/build/snemo/Bayeux.build/;/vols/build/snemo/geant4.9.6.p04.build/;/vols/build/snemo/boost_1_63_0" ../Falaise
make
make install

Test:

export LD_LIBRARY_PATH=/vols/build/snemo/camp.build/
export G4DATA=/vols/build/snemo/geant4.9.6.p04/data/
export G4LEDATA=${G4DATA}/G4EMLOW6.32

/vols/build/snemo/Falaise.build/bin/flsimulate -c ~/SuperNEMO/MC/mysimulation.conf -o test.brio
/vols/build/snemo/Falaise.build/bin/flreconstruct -i test.brio -p /vols/build/snemo/Falaise.build/share/Falaise-4.0.1/resources/snemo/demonstrator/reconstruction/official-2.0.0.conf -o reco.brio