Migrate to CMake

[villaa]

I am submitting a feature request.

The feature I am requesting is for: make/CI tests

I am requesting: Migrating to CMake--a more modern build system; this will also allow testing to be simplified.

Ideas for implementation: see the "obscura" code by temken on github.

[nuclearGoblin]

Link to the aforementioned Obscura code: https://github.com/temken/obscura/blob/master/.github/workflows/main.yml Link to sample results for Obscura's testing through GitHub: https://github.com/temken/obscura/runs/4345220111?check_suite_focus=true Link to the Obscura repository main page: https://github.com/temken/obscura

[villaa]

Solving this issue will obviate issue #26 because CMake is a more modern replacement for automake tools.

[villaa]

Having real headaches getting CMake to work--doing some low-level debugging on the CMakeLists.txt because we are getting binaries with shared object files not found even though they clearly exist. Only on Mac OSX

See this page to learn what the -isysroot argument does to to the Mac compiler suite c++

[villaa]

One of the most confounding things is that the failure happens only for Mac OSX and only on Travis (local builds seem to work fine).

Here is the compile line for realizeCascades on Travis:

/Applications/Xcode-11.4.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -D__GIT_VERSION="\"8a81\"" -fprofile-arcs -ftest-coverage -isysroot /Applications/Xcode-11.4.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -coverage CMakeFiles/realizeCascades.dir/bin/realizeCascades.cpp.o -o realizeCascades -Wl,-rpath,/Users/travis/build/nrCascadeSim_build/lib weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libCore.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libImt.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libRIO.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libNet.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libHist.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libGraf.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libGraf3d.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libGpad.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libROOTDataFrame.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libTree.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libTreePlayer.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libRint.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libPostscript.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libMatrix.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libPhysics.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libMathCore.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libThread.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libMultiProc.so lib/libncap.dylib /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libCore.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libImt.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libRIO.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libNet.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libHist.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libGraf.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libGraf3d.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libGpad.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libROOTDataFrame.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libTree.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libTreePlayer.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libRint.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libPostscript.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libMatrix.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libPhysics.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libMathCore.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libThread.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libMultiProc.so weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o

And then here is the error we get upon running realizeCascades

$ ./test_realizeCascades.sh
dyld: Library not loaded: @rpath/libCore.6.20.so
  Referenced from: /usr/local/bin/realizeCascades

WHAT?! Those files are explicitly listed on the linking command above and the files exist, running ls -lh returns:

lrwxr-xr-x 1 travis staff 15B Feb 26 2020 /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libCore.so -> libCore.6.20.so

This is referencing this build.

[villaa]

Incidentally this is the compile line that works on my Mac:

/usr/local/Cellar/cmake/3.22.1/bin/cmake -E cmake_link_script CMakeFiles/realizeCascades.dir/link.txt --verbose=1 /Library/Developer/CommandLineTools/usr/bin/c++ -D__GIT_VERSION="\"v1.3.2-69-g4a1f\"" -fprofile-arcs -ftest-coverage -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -coverage CMakeFiles/realizeCascades.dir/bin/realizeCascades.cpp.o -o realizeCascades -Wl,-rpath,/Users/villaa/nrCascadeSim_build/lib weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o /usr/local/lib/root/libCore.so /usr/local/lib/root/libImt.so /usr/local/lib/root/libRIO.so /usr/local/lib/root/libNet.so /usr/local/lib/root/libHist.so /usr/local/lib/root/libGraf.so /usr/local/lib/root/libGraf3d.so /usr/local/lib/root/libGpad.so /usr/local/lib/root/libROOTDataFrame.so /usr/local/lib/root/libTree.so /usr/local/lib/root/libTreePlayer.so /usr/local/lib/root/libRint.so /usr/local/lib/root/libPostscript.so /usr/local/lib/root/libMatrix.so /usr/local/lib/root/libPhysics.so /usr/local/lib/root/libMathCore.so /usr/local/lib/root/libThread.so /usr/local/lib/root/libMultiProc.so /usr/local/lib/root/libROOTVecOps.so lib/libncap.dylib /usr/local/lib/root/libCore.so /usr/local/lib/root/libImt.so /usr/local/lib/root/libRIO.so /usr/local/lib/root/libNet.so /usr/local/lib/root/libHist.so /usr/local/lib/root/libGraf.so /usr/local/lib/root/libGraf3d.so /usr/local/lib/root/libGpad.so /usr/local/lib/root/libROOTDataFrame.so /usr/local/lib/root/libTree.so /usr/local/lib/root/libTreePlayer.so /usr/local/lib/root/libRint.so /usr/local/lib/root/libPostscript.so /usr/local/lib/root/libMatrix.so /usr/local/lib/root/libPhysics.so /usr/local/lib/root/libMathCore.so /usr/local/lib/root/libThread.so /usr/local/lib/root/libMultiProc.so /usr/local/lib/root/libROOTVecOps.so weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o

It looks basically the same in the way that it references the relevant ROOT libraries, like libCore.so, what gives?

[villaa]

Still working on this-- really annoying, cannot reproduce what I see on Travis. I tried a whole new directory on my Macbook Pro Mojave (10.14) and did the whole process. Eventually I get:

/Library/Developer/CommandLineTools/usr/bin/c++ -D__GIT_VERSION="\"v1.3.2-76-gfd1e\"" -fprofile-arcs -ftest-coverage -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -coverage CMakeFiles/realizeCascades.dir/bin/realizeCascades.cpp.o -o realizeCascades -Wl,-rpath,/Users/villaa/travis_nrCascadeSim/nrCascadeSim_build/lib -Wl,-rpath,/Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o lib/libncap.dylib weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o /Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib/libTree.6.20.00.so /Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib/libImt.6.20.00.so /Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib/libNet.6.20.00.so /Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib/libRIO.6.20.00.so /Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib/libThread.6.20.00.so /Users/villaa/travis_nrCascadeSim/nrCascadeSim/root/lib/libCore.6.20.00.so

So, very simillar to the latest Travis:

/Applications/Xcode-11.4.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -D__GIT_VERSION="\"fd1e\"" -fprofile-arcs -ftest-coverage -isysroot /Applications/Xcode-11.4.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -coverage CMakeFiles/realizeCascades.dir/bin/realizeCascades.cpp.o -o realizeCascades -Wl,-rpath,/Users/travis/build/nrCascadeSim_build/lib -Wl,-rpath,/Users/travis/build/villano-lab/nrCascadeSim/root/lib weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o lib/libncap.dylib weisskopf.o lindhard.o cascadeProd.o edepmath.o rootUtil.o /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libTree.6.20.00.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libImt.6.20.00.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libNet.6.20.00.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libRIO.6.20.00.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libThread.6.20.00.so /Users/travis/build/villano-lab/nrCascadeSim/root/lib/libCore.6.20.00.so

but the one on my Mac works!!?

[villaa]

The only difference(s) are that on my Macbook it gets the right __GIT_VERSION and on Travis it does not. On Travis it only get's the part after the dash, fd1e. Why?

Also on my Mac I accidentally used the wrong downloaded ROOT binary and as a result the MacOSX10.15.sdk was searched for but not found. But this only resulted in this warning:

warning: no such sysroot directory: '/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk'

but the program realizeCascades ran correctly anyway.

[villaa]

... and if I download the correct ROOT binary for my system, namely root_v6.20.00.macosx64-10.14-clang110.tar.gz instead of root_v6.20.00.macosx64-10.15-clang110.tar.gz it works on my Macbook without that errror.

AND I found out that travis was only using fd1e for the version b/c it has a different version of git (2.26.2) where the --abbrev=4 means something different?? As opposed to the newer versions (2.35.0). This is sort of a guess.

Actually this last bit is WRONG, the linux builds on Travis use github 2.35.0 and still only give the last 4 digits in response to git describe --abbrev=4 --always whereas on my Macbook (which reports to have the same git version) gives v1.3.2-79-g550d I have no idea what's going on with that.

[villaa]

AH HA! when I try to run realizeCascades after a global installation at /usr/local/bin and calling it from inside a second script test_realizeCascades.sh then I get this error on my Macbook Pro:

dyld: Library not loaded: @rpath/libTree.6.20.so
  Referenced from: /usr/local/bin/realizeCascades
  Reason: image not found
./test_realizeCascades.sh: line 4: 27887 Abort trap: 6           realizeCascades -n 100000 -o "output.root?reproducible=fixedname" -d 1 ../levelfiles/Si28_ngam_all_cascades_rfmt_sorted.txt
Traceback (most recent call last):
  File "/Users/villaa/travis_nrCascadeSim/nrCascadeSim/tests/./isempty.py", line 1, in <module>
    import uproot
ModuleNotFoundError: No module named 'uproot'

[villaa]

When I go ahead and switch to a conda environment that has uproot I get:

(nrCascadeSim) Anthonys-MacBook-Pro-2:tests villaa$ ./test_realizeCascades.sh 
dyld: Library not loaded: @rpath/libTree.6.20.so
  Referenced from: /usr/local/bin/realizeCascades
  Reason: image not found
./test_realizeCascades.sh: line 4: 27922 Abort trap: 6           realizeCascades -n 100000 -o "output.root?reproducible=fixedname" -d 1 ../levelfiles/Si28_ngam_all_cascades_rfmt_sorted.txt
Traceback (most recent call last):
  File "/Users/villaa/anaconda3/envs/nrCascadeSim/lib/python3.8/site-packages/uproot/source/file.py", line 113, in _open
    self._file = numpy.memmap(self._file_path, dtype=self._dtype, mode="r")
  File "/Users/villaa/anaconda3/envs/nrCascadeSim/lib/python3.8/site-packages/numpy/core/memmap.py", line 225, in __new__
    f_ctx = open(os_fspath(filename), ('r' if mode == 'c' else mode)+'b')
FileNotFoundError: [Errno 2] No such file or directory: 'output.root'

Which seems consistent with the Travis Mac errors:

$ ./test_realizeCascades.sh
dyld: Library not loaded: @rpath/libTree.6.20.so
  Referenced from: /usr/local/bin/realizeCascades
  Reason: image not found
./test_realizeCascades.sh: line 4:  3321 Abort trap: 6           realizeCascades -n 100000 -o "output.root?reproducible=fixedname" -d 1 ../levelfiles/Si28_ngam_all_cascades_rfmt_sorted.txt
Traceback (most recent call last):
  File "/Users/travis/build/villano-lab/nrCascadeSim/tests/./isempty.py", line 2, in <module>
    file = uproot.open('output.root') #Change this to change tested file.
  File "/Users/travis/miniconda/envs/piptest/lib/python3.9/site-packages/uproot/rootio.py", line 54, in open
    return ROOTDirectory.read(openfcn(path), **options)
  File "/Users/travis/miniconda/envs/piptest/lib/python3.9/site-packages/uproot/rootio.py", line 51, in <lambda>
    openfcn = lambda path: MemmapSource(path, **kwargs)
  File "/Users/travis/miniconda/envs/piptest/lib/python3.9/site-packages/uproot/source/memmap.py", line 21, in __init__
    self._source = numpy.memmap(self.path, dtype=numpy.uint8, mode="r")
  File "/Users/travis/miniconda/envs/piptest/lib/python3.9/site-packages/numpy/core/memmap.py", line 228, in __new__
    f_ctx = open(os_fspath(filename), ('r' if mode == 'c' else mode)+'b')
FileNotFoundError: [Errno 2] No such file or directory: 'output.root'

[villaa]

I found that on the Mac systems, strangely, the executable realizeCascades works from the command line but does not when called from within a script. When executed from within a script it leads to the above library error.

I stumbled on this obscure matlab post. That lead me to this post.

This suggests that DYLD_LIBRARY_PATH is not inherited in a daughter shell and needs to be set on the same line as the executable run in the shell. I made this change and it seems we are passing tests on the CMake branch.

[villaa]

PR #91 finally worked after being able to reproduce the error locally (see above) issue closed.