Allow installation of dependencies via conda

[tkittel]

This might already be possible, but we should try to find a bullet proof recipe. For now I am collecting notes in this issue.

[tkittel]

I have a feeling that in principle this will be very easy, if we completely skip the whole dgdepfixer issue. However, to do that we need to work with vanilla versions of e.g. Geant4 and OpenSceneGraph which might complicate matters. At the very least we need to update Geant4 to a more recent versions (cf. #2)

[tkittel]

Playing around, here is an environment.yml which I think works (it needs more testing though):

name: dgcode
channels:
  - nodefaults
  - conda-forge
dependencies:
  - python=3.10
  - clang=14.0.6
  - clangxx=14.0.6
  - compilers
  - numpy
  - matplotlib
  - cmake
  - git
  - gdb
  - valgrind
  - pip
  - pip:
    - ptpython==3.0.20
variables:
  DGCODE_FORCEVAR_CC: clang
  DGCODE_FORCEVAR_CXX: clang++
  DGCODE_BOOTSTRAP_SKIP_DGDEPFIXER_CHECK: 1
  URL_DGCODE: git@github.com:mctools/dgcode.git
  URL_DGCODE_VAL: git@github.com:mctools/dgcode_val.git
  URL_DGCODE_DG_PROJECTS: git@github.com:ess-dg/dgcode_projects.git
  URL_DGCODE_DG_PRIVATE_PROJECTS: git@github.com:mctools/dgcode_private_projects.git

It needs DGCODE_BOOTSTRAP_SKIP_DGDEPFIXER_CHECK support in bootstrap.sh and different python detection, which is WIP in the tk_devel branch. (added to main)

I also didn't add geant4/openscenegraph/root to the list yet. For Geant4 it might not work unless we migrate to the latest version.

As a test (because I was debugging an unrelated clang14 issue) the .yml above uses 4 lines to enable usage of the clang14 compiler, obviously those 4 lines should in general be left out. I also pinned the python version for similar reasons.

(update: replaced CC/CXX with DGCODEFORCEVAR... versions, see comment below)

[tkittel]

Update: The CC and CXX variables in the .yml above does not actually get set (rather: they get overridden somehow). Perhaps we need DGCODE_FORCEVAR_CC / DGCODE_FORCEVAR_CXX for bootstrap.sh to restore the CC/CXX? update: this is now supported

[tkittel]

Trying again with the .yml above, but this time with clang=10, clang++=10, python=3.8, I ran into problem during the link stage of packages (not on DGBoost, Core, or Utils, but on later packages like MCPL). It was solved if I added $CONDA_PREFIX/lib to my LD_LIBRARY_PATH. Perhaps we need to add rpath to that dir when conda is detected?

update: the same issue also happened in a different environment where the clang/python related lines were absent. So in general we need to add the CONDA_PREFIX/lib to LD_LIBRARY_PATH, or presumably, to rpath.

[tkittel]

Today I was trying to be a bit less ambitious and simply try to set up a minimal conda environment in which to configure and build a standalone Geant4 example (geant4 example B1). So I tried:

name: dgcode
channels:
  - nodefaults
  - conda-forge
dependencies:
  - cmake
  - compilers
  - geant4
  - expat
variables:
  - DGCODE_BOOTSTRAP_SKIP_DGDEPFIXER_CHECK: 1

The compilers package seems to be crucial, as it drags in all of sorts of basic system headers: https://github.com/conda-forge/compilers-feedstock/blob/main/recipe/meta.yaml

The expat package was needed for some reason, possibly the geant4-feedstock recipe has it missing in the run section, not sure.

With all that, a naive CMake attempt (on the B1 example) still fails:

  Could NOT find EXPAT: Found unsuitable version "2.2.9", but required is at
  least "2.5.0" (found /usr/lib/x86_64-linux-gnu/libexpat.so)
Call Stack (most recent call first):
  /home/tkittel/scr/miniforge_install/miniforge_installed/envs/g411_standalone/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:592 (_FPHSA_FAILURE_MESSAGE)
  /home/tkittel/scr/miniforge_install/miniforge_installed/envs/g411_standalone/share/cmake-3.24/Modules/FindEXPAT.cmake:65 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /home/tkittel/scr/miniforge_install/miniforge_installed/envs/g411_standalone/share/cmake-3.24/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /home/tkittel/scr/miniforge_install/miniforge_installed/envs/g411_standalone/lib/Geant4-11.0.3/Geant4Config.cmake:268 (find_dependency)
  CMakeLists.txt:15 (find_package)

because cmake is not looking correctly inside the $CONDA_PREFIX. Obviously we need to pass some variables, as in the '$CMAKE_ARGSused in conda recipesbuild.sh` scripts.

update: investigating a bit further, I realised that I now suddenly seems to have the CMAKE_ARGS env var set, possibly because I re-activated my conda environment AFTER installing the compilers meta-package. So inside conda, we should be sure to 1) always have the compilers dependency and 2) invoke cmake ... as cmake ${CMAKE_ARGS} ...

[tkittel]

Summary of lessons learnt today:

• The compiler meta package must be installed
• The conda-forge geant4 package seems to be missing expat as a dependency, unless it was a red herring caused by the other issues (to check).
• The CMAKE_ARGS env var must be used (todo: make this available via dgbuild.cfg ?)
• The LDFLAGS must be propagated from our cmake layer and applied. In CMake these are used to populate CMAKE_SHARED_LINKER_FLAGS_INIT and CMAKE_EXE_LINKER_FLAGS_INIT. For now I will add a check that these are always identical, and simply tack these linker flags onto our global variable.
• We should check that we also use the CFLAGS variable correctly.

[tkittel]

Tests like ess_mcpltests_testflags probably need to be conda-aware (or at least look at LDFLAGS etc.) as well.

[tkittel]

ess_skeletonsp_sim -n1 seg-faults, not sure if it is g4 or conda related. In any case, I should check there is also a unit test for this command. update:

==61867== Invalid read of size 8
 504 ==61867==    at 0x7731792: G4GSMottCorrection::GetMottCorrectionFactors(double, double, int, double&, double&, double&) (in /home/tkittel/scr/miniforge_install/miniforge_insta\
     lled/envs/g4_v11/lib/libG4processes.so)

update: this was a bug in G4 (see #2).

[tkittel]

Related to the issue of $CONDA_PREFIX/lib needing to be added to LD_LIBRARY_PATH, is the issue that we do not recompile automatically if changing the LD_LIBRARY_PATH (not even if running with dgbuild -i).

[tkittel]

I think we need to look for $CONDA_PREFIX in bootstrap.sh and dgbuild, and make them conda-aware:

• Perhaps (or not, see next item) Make sure $CONDA_PREFIX/lib is added to (DY)LD_LIBRARY_PATH (bootstrap.h).
• Make sure $CONDA_PREFIX/lib is added to link-time and runtime rpath's.
• Make sure cmake invocations gets the ${CMAKE_ARGS} added as appropriate. As a bonus, make sure this also happens for any cmake invocations in dgbuild scripts (i.e. ess_mcplexport_testcmake etc.).

[tkittel]

Working on this again. Adding a list here with notes for myself:

• conda-build had issue building local recipes on my machine (e.g. the ncrystal-feedstock/recipe), but installing conda-build into base helped. Might be related to https://github.com/conda/conda/issues/12955.
• To avoid going crazy: conda update -n base conda + conda install -n base conda-libmamba-solver + conda config --set solver libmamba (undo with --set solver classic)
• root (even root_base) conflicts with g4. Specifically this might (might because it is a huge amount of deps) be krb5 (from xrootd in root) conflicting. Solution might be to provide a custom root package in minimal mode.
• We need to update g4 version to latest before we can work out of conda.
• Big issue is that we want to replace NCrystalRel with ncrystal from conda. However, the code in ncrystaldev has clashing symbols. We might have to add option for different symbols etc. in ncrystaldev?
• Also consider mcpl, since both us and e.g. McStas might want the mcpl conda package installed.