CMake warnings on mcconfigurator

[YanzhaoW]

Describe the bug When configuring FairRoot CMake in a conda environment, following warning is given by CMake:

CMake Warning at fairroot/mcconfigurator/CMakeLists.txt:17 (add_library):
  Cannot generate a safe runtime search path for target MCConfigurator
  because files in some directories may conflict with libraries in implicit
  directories:

    runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
      /u/yanwang/miniconda3/lib
    runtime library [libexpat.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
      /u/yanwang/miniconda3/lib

  Some of these libraries may not be found correctly.

System information (please complete the following information):

• OS: Debian Buster
• Compiler: gcc 13
• Environment: FairSoft latest, FairRoot latest

[dennisklein]

So, CMake finds libz and libexpat in /usr/lib/x86_64-linux-gnu and makes a sanity check with your run-time ld config which provides those libs in /u/yanwang/miniconda3/lib - and warns you about it. To fix this, you could try pointing CMake to your miniconda3 libs via -DZLIB_ROOT=/u/yanwang/miniconda3 -DEXPAT_ROOT=/u/yanwang/miniconda3 or alternatively -DCMAKE_PREFIX_PATH=/u/yanwang/miniconda3. Or, if you want to use the system libs, unload/modify your conda environment.

[YanzhaoW]

I see. So if I do nothing, CMake would still use the system libs for libz and libexpat?

[dennisklein]

The question is what is your intention with putting those miniconda libs in your run-time config but different ones in your build time config. CMake is just warning you after all. If you do this intentionally, then you have your reasons, no?

[YanzhaoW]

Sorry, I don't quite understand it.

By "run-time config", I guess you meant the configuration set with LD_LIBRARY_PATH and PATH env variables I use to build FairRoot. These two env variables are appended with the conda path automatically when I enable the environment.

With "build time config", do you mean the LD_LIBRARY_PATH and PATH config when I built the FairSoft?

[fuhlig1]

The question probably is why do you need the miniconda environment when installing FairRoot.

[dennisklein]

   LD_LIBRARY_PATH
         A list of directories in which to search for ELF libraries
         at execution time.  The items in the list are separated by
         either colons or semicolons, and there is no support for
         escaping either separator.  A zero-length directory name
         indicates the current working directory.
         (...)

https://man7.org/linux/man-pages/man8/ld.so.8.html subsection Environment Variables

With "build time config", do you mean the LD_LIBRARY_PATH and PATH config when I built the FairSoft?

No, $LD_LIBRARY_PATH is only used at run-time - it is a library search path for the dynamic linker. $PATH is again a completely different search path which is typically used by your shell to find commands which you specify without any file path.

---

• By activating your miniconda environment, I suspect, it sets $LD_LIBRARY_PATH.
• This has the effect that all programs that run in this shell session resolve their dynamic library dependencies including the entries added by miniconda.
• It appears /u/yanwang/miniconda3/lib contains libz and libexpat.
• However, CMake does not look at $LD_LIBRARY_PATH, so it will find libz and libexpat in your system directories (system dirs are searched last by default).
• It appears, that CMake has a sanity check though, that looks at what your run-time config will resolve those libraries to compared to what itself resolved it to and generates a warning since, usually, those are matching.
• You now have three choices basically:
  1. You know the miniconda libz and libexpat are compatible with your system ones. Nothing to do.
  2. You do not want to use the miniconda-provided libz and libexpat, then modify your miniconda environment to not include those
     • Setting $LD_LIBRARY_PATH is global to everyone, there is also the option to bake a RUNPATH into the executables/library files. So, in case your miniconda libz and libexpat are just some transitive dependency of something in your miniconda environment, it might be an option to modify the miniconda recipes in such a way that it generates RUNPATH entries instead of setting $LD_LIBRARY_PATH (the latter is usually bad practice - yes, we do it as well in FairRoot, but this needs to be improved upon)
  3. You do want to use the miniconda-provided libz and libexpat, then tell CMake where to find those (see my first post).

[YanzhaoW]

Thanks for your detailed explanation. It clears up many things for me.

I checked again with my conda environment. It actually doesn't set $LD_LIBABRAY_PATH at all. The only changed relevant env variable I could see is $PATH.

So I open the CMakeCache.txt to see how my miniconda3/lib sneaks into the CMake configuration. Then I found out that it is yaml-cpp package and CMake somehow find the one that I installed from the Conda environment. Knowing this, I went to CMake website to check how CMake finds required packages:

Search the standard system environment variables. This can be skipped if NO_SYSTEM_ENVIRONMENT_PATH is passed or by setting the CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH to FALSE. Path entries ending in /bin or /sbin are automatically converted to their parent directories: PATH

So CMake does use $PATH environment during the configuration to find yaml-cpp package. The yaml-cpp installed in Conda is naturally linked to zlib in miniconda3/lib while the same target MCConfigurator is also linked to another zlib either through root or geant4vmc.

Therefore, simply specifying zlib path with -DZLIB_ROOT=/u/yanwang/miniconda3 -DEXPAT_ROOT=/u/yanwang/miniconda3 for CMake doesn't work and I got more warnings. The solution should be explicitly set the yaml package path for CMake instead:

cmake -Dyaml-cpp_ROOT=/usr/lib/x86_64-linux-gnu/cmake/yaml-cpp ..

But should we put yaml-cpp into FairSoft such that CMake could find it using ${CMAKE_PREFIX_PATH}?

[dennisklein]

I don't follow in all details. Can you share your CMakeCache.txt or at least the entries for expat and zlib? (when the original warning happens without specifying additional hints as discussed here)

[fuhlig1]

@YanzhaoW,

I think your findings are correct. Geant4VMC and Geant4 libraries link against libz and libexpat which in your case where found in the system. When compiling FairRoot geant4vmc and yaml-cpp are required as dependencies for MCConfigurator which come with conflicting requirements for libz and and libexpat.

Did you use your miniconda environment also when compiling FairSoft?

[YanzhaoW]

Hi, @dennisklein

Here is the CMakeCache.txt generated in conda environment: CMakeCache.txt

[YanzhaoW]

Hi, @fuhlig1

I don't think I did. But it would be nice if FairSoft also contains yaml-cpp by itself and then everything would be compatible with each other. There will be always some problems if we just let CMake find yaml-cpp in whatever way.