Sparrow Build Failure due to unlinked yaml-cpp

Dear Rehier Group,

Thanks for helping earlier with Utilities. I've been debugging and reached as far as the Sparrow install. The puffin bootstrap has installed Core, Utilities, Readuct and Database but when trying to install Sparrow 5.0.0 it is unable to find yaml-cpp.

yaml-cpp is install as: $ ls /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/libyaml-cpp.so libyaml-cpp.so libyaml-cpp.so.0.7 libyaml-cpp.so.0.7.0

I've tried adding the full path to the -RPATH as it suggests by including the cmake flag: -DCMAKE_EXE_LINKER_FLAGS="-Wl,-rpath,/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/"

Any ideas as to what I've done wrong would be very helpful

OS is RedHat CentOS7. Output log is (after installing other packages via python3 -m puffin -c puffin.yaml bootstrap:

Preparing Sparrow...

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example:

git switch -c

Or undo this operation with:

git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 51cbda5 Merge branch 'develop' for release 1.2.2 CMake Warning (dev) at build/cereal-src/CMakeLists.txt:2 (project): Policy CMP0048 is not set: project() command manages VERSION variables. Run "cmake --help-policy CMP0048" for policy details. Use the cmake_policy command to set the policy and suppress this warning.

The following variable(s) would be set to empty:

PROJECT_VERSION
PROJECT_VERSION_MAJOR
PROJECT_VERSION_MINOR
PROJECT_VERSION_PATCH

This warning is for project developers. Use -Wno-dev to suppress it.

-- Cereal was not found in your PATH, so it was downloaded. -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- Found OpenMP: TRUE (found version "4.5")
-- Found Boost: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/cmake/Boost-1.74.0/BoostConfig.cmake (found version "1.74.0") found components: program_options filesystem -- Found PythonInterp: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/python3 (found version "3.8.18") -- Cereal found locally at cereal_DIR-NOTFOUND -- Targets to copy for python bindings: Sparrow;Scine::UtilsOS -- Found PY_sphinx: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/sphinx
-- Configuring done -- Generating done -- Build files have been written to: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/sparrow/build Scanning dependencies of target SparrowApp [ 1%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/CalculationHandler.cpp.o [ 2%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/CommandLineOptions.cpp.o [ 2%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/SparrowInitializer.cpp.o [ 3%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/main.cpp.o [ 4%] Linking CXX executable sparrow /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libyaml-cpp.so.0.7, needed by /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so, not found (try using -rpath or -rpath-link) /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::StartedScalar()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference tovtable for YAML::BadConversion' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::insert_map_pair(YAML::detail::node&, YAML::detail::node&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::ostream_wrapper::write(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::memory::create_node()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::Write(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::set_null()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference tovtable for YAML::InvalidNode' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to typeinfo for YAML::InvalidNode' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::detail::node_data::set_scalar(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::end()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::~Emitter()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::memory_holder::merge(YAML::detail::memory_holder&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::detail::node_data::size() const' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::empty_scalar[abi:cxx11]()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::good() const' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::InvalidNode::~InvalidNode()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference totypeinfo for YAML::BadSubscript' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::PrepareNode(YAML::EmitterNodeType::value)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference totypeinfo for YAML::BadConversion' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::SetMapFormat(YAML::EMITTER_MANIP)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::detail::node_data::convert_to_map(std::shared_ptr const&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::PrepareIntegralStream(std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&) const' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::Write(bool)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::begin()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::SetSeqFormat(YAML::EMITTER_MANIP)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::BadConversion::~BadConversion()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::c_str() const' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to vtable for YAML::RepresentationException' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::Emitter()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::BadSubscript::~BadSubscript()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference tovtable for YAML::Exception' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::mark_defined()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference toYAML::Emitter::SetLocalValue(YAML::EMITTER_MANIP)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::convert<bool>::decode(YAML::Node const&, bool&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference tovtable for YAML::BadSubscript' collect2: error: ld returned 1 exit status make[2]: [src/Sparrow/sparrow] Error 1 make[1]: [src/Sparrow/CMakeFiles/SparrowApp.dir/all] Error 2 make: *** [all] Error 2 Traceback (most recent call last): File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/main__.py", line 148, in main() File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/main.py", line 134, in main main_bootstrap(config) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/main.py", line 73, in main_bootstrap bootstrap(config) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/bootstrap.py", line 85, in bootstrap program.install(program_build_dir, install_dir, config["resources"]["cores"]) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/programs/sparrow.py", line 22, in install self.scine_module_install(repo_dir, install_dir, ncores, add_lib=True, add_bin=True) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/programs/program.py", line 162, in scine_module_install subprocess.run(["make", "-j" + str(ncores), "install"], env=env, check=True) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['make', '-j1', 'install']' returned non-zero exit status 2.

Based on your output in https://github.com/qcscine/utilities/issues/7 which was:

 -- Found package yaml-cpp at /home/njh80/modulesSource/miniforge3/envs/chemoton/share/cmake/yaml-cpp

you have a similar problem as with the Core dependency. Only that here the problem is not a different version, but that Sparrow cannot find the dependency with which Utils has been built. It should work if you add the directory where libyaml.so is located (probably /home/njh80/modulesSource/miniforge3/envs/chemoton/lib to your LD_LIBRARY_PATH, but that could get you back to problems like the one in https://github.com/qcscine/utilities/issues/7

Due to the dependencies between our modules, it is a bit difficult to mix in dependencies between different locations without setting up the environment carefully. In general, we recommend: only install boost, mongo-c, and mongo-cxx separately and then our bootstrap should be able to download and compile all dependencies and configure everything into one install directory.

So my hotfix recommendation would be: remove the yaml-cpp location from all your paths, delete the utils build directory (/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/utils) and start the bootstrap again. If the cmake of utils shows the message:

-- yaml-cpp was not found in your PATH, so it was downloaded.

and then first compiles the yaml-cpp library (noticeable by having more warnings printed), then you should have it properly installed within the puffin bootstrap and Sparrow should link fine.

Simply adding the yaml location to the LD_LIBRARY_PATH would be the faster solution if it works straightaway, because you don't need to compile utils again, but you need to be careful to always add this to your path if you want to use our software and this dependency will not be added to the puffin.sh file generated after the bootstrap

Thanks again @steinmig! Looks like the LD_LIBRARY_PATH did the trick here. I'm not sure how yaml-cpp was installed into the virtual environment originally, maybe Conda installed it with gcc?

I managed to get a bit further with the bootstrapping up to Molassembler, but then there was a different problem. It's probably a similar problem but I thought I'd just add it to this thread (I can setup a fresh query if preferred).

This time Nauty isn't downloading. I've got a copy of nauty (2.8.6) elsewhere but it is not in the PATH, LD_LIBRARY_PATH, or CPATH. The identical error occurs even when I include my nauty on the PATH (I assume it is looking for 2.7.0).

Preparing Molassembler...

-- The C compiler identification is GNU 11.4.0 -- The CXX compiler identification is GNU 11.4.0 -- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc -- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc - works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ -- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ - works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Scine::UtilsOS found locally at /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/cmake/ScineUtilsOS -- Enabling GCC specific warning flags -- Found MKL: /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include
-- Found Intel MKL libraries: -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_avx2.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_vml_avx2.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_core.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_intel_lp64.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_gnu_thread.so -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- Found OpenMP: TRUE (found version "4.5")
-- Found MKL for use with Eigen3 CMake Error at dev/cmake/ColorMessages.cmake:39 (message): Could not download file: "Unsupported protocol" Please download the file at https://scine.ethz.ch/static/download/nauty27r1.tar.gz and place it in your build tree at /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/nauty27r1.tar.gz Call Stack (most recent call first): dev/cmake/DownloadFileHelper.cmake:18 (cmessage) cmake/ImportNauty.cmake:23 (download_file) CMakeLists.txt:77 (import_nauty)

-- Configuring incomplete, errors occurred!

I can do wget to the https fine in the folder but every time I rebootstrap Puffin I need to clear out the existing software to avoid the error:

File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/programs/program.py", line 121, in scine_module_install repository.remotes.origin.pull() File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/git/remote.py", line 1068, in pull res = self._get_fetch_info_from_stderr(proc, progress, kill_after_timeout=kill_after_timeout) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/git/remote.py", line 858, in _get_fetch_info_from_stderr proc.wait(stderr=stderr_text) File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/git/cmd.py", line 657, in wait raise GitCommandError(remove_password_if_present(self.args), status, errstr) git.exc.GitCommandError: Cmd('git') failed due to: exit code(1) cmdline: git pull -v -- origin

Maybe editing the bootstrap.py so that it can clear and continue interrupted processes might help? For example:

core = Utils(config.programs()["core"]) if not os.path.isfile(install_dir + "/lib/libcore.so"): shutil.rmtree(core_build_dir, ignore_errors=True) core.install(core_build_dir, install_dir, config["resources"]["cores"])

According to https://stackoverflow.com/questions/29816529/unsupported-protocol-while-download-tar-gz-package and similar search results, the problem is that your cmake installation does not support SSL.

A possible quick fix would be:

Download the file and place it as instructed
Instead of starting the bootstrap, go to /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/ and execute:
- cmake ..
- make -j4 install This might compile and install Molassembler together with the library correctly. Then you could try to relaunch the boostrap, which should hopefully detect that Molassembler is configured and installed, or detects now the installed nauty library and does not require the download. The later might require you to set: export LD_LIBRARY_PATH=/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib:/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib64:$LD_LIBRARY_PATH
Again, this is just a hot fix and might complicate things, if you are updating in the future. The cleaner version is to load/install a cmake version with SSL support

Hi @steinmig, sorry for not getting back sooner. I've been having a go and essentially the way bootstrap of Puffin is coded it doesn't try to check if modules are already there first. It just starts the build in the same folder and evidently errors when it realises the things are already there, so that would need a change to the puffin bootstrap method.

I've got around this by editing the puffin module - I'd recommend it as a general fix. However, the module still causes some trouble. I've also had to manually pip install . in each python module because the puffin pip call was installing in the wrong place.

The CMake version should have SSL support but I didn't install this and I'm finding an install of a newer CMake a tad difficult.

Any ideas why the created in situ modules aren't happy? My understanding is that the file ImportNauty.cmake is meant to build a file nautyTargets.cmake but that this is being ignored. CMakeCache.txt ImportNauty.txt

-- The C compiler identification is GNU 11.4.0 -- The CXX compiler identification is GNU 11.4.0 -- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc -- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc - works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ -- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ - works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Scine::UtilsOS found locally at /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/cmake/ScineUtilsOS -- Enabling GCC specific warning flags -- Found MKL: /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include
-- Found Intel MKL libraries: -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_avx2.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_vml_avx2.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_core.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_intel_lp64.so -- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_gnu_thread.so -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- Found OpenMP: TRUE (found version "4.5")
-- Found MKL for use with Eigen3 CMake Error at build/nauty/nautyConfig.cmake:1 (include): include could not find load file:

/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/nauty/nautyTargets.cmake

Call Stack (most recent call first): cmake/ImportNauty.cmake:12 (find_package) CMakeLists.txt:77 (import_nauty)

CMake Error at build/RingDecomposerLib/RingDecomposerLibConfig.cmake:1 (include): include could not find load file:

/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/RingDecomposerLib/RingDecomposerLibTargets.cmake

Call Stack (most recent call first): cmake/ImportRingDecomposerLib.cmake:11 (find_package) CMakeLists.txt:83 (import_rdl)

-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success -- Performing Test COMPILER_HAS_DEPRECATED_ATTR -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success -- pybind11 v2.10.4 -- Found PythonInterp: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/python3 (found suitable version "3.8.18", minimum required is "3.6") -- Found PythonLibs: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/libpython3.8.so -- Performing Test HAS_FLTO -- Performing Test HAS_FLTO - Success -- Pybind11 was not found in your PATH, so it was downloaded. -- Found PY_pytest: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/pytest
-- Found PY_scine_utilities: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_utilities/scine_utilities.cpython-38-x86_64-linux-gnu.so
-- Could NOT find PY_pybind11_stubgen (missing: PY_PYBIND11_STUBGEN) -- Not generating typechecking stubs for python package as pybind11-stubgen was not found -- Found PY_pip: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/pip
-- Found PY_sphinx: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/sphinx
-- Configuring incomplete, errors occurred! See also "/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/CMakeFiles/CMakeOutput.log".

qcscine / sparrow

Sparrow Build Failure due to unlinked yaml-cpp #25