search

ValeevGroup / tiledarray

A massively-parallel, block-sparse tensor framework written in C++
GNU General Public License v3.0
247 stars 51 forks source link

Various CMake + `ENABLE_CUDA` Fixes #415

Closed wavefunction91 closed 11 months ago

wavefunction91 commented 11 months ago

Fix various CMake bugs stemming from ENABLE_CUDA

  1. LibRETT used to fail with parallel build + GNU Makefiles (ala the Umpire build bug and fix). This fix follows the prescription used in #340.
  2. Unsure how this ever worked, but for some reason (at least for CMake 3.25+), you need to explicitly link to the CUDA::cudart TARGET, because apparently it doesn't get propagated via other dependencies... Unclear why CUDA::cublas doesn't propagate it, but who knows. Before this fix, I get the following on my system (CUDA 11.17 CMake 3.25) when building and linking ta_test 
    [ 88%] Linking CXX executable ta_test
cd /home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/tests && /home/dbwy/Software/BuildSystems/CMake/3.25.0/install/bin/cmake -E cmake_link_script CMakeFiles/ta_test.dir/link.txt --verbose=1
/usr/bin/c++ -g -Wl,-rpath -Wl,/home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/objdir/../install/lib -Wl,--enable-new-dtags CMakeFiles/ta_test.dir/ta_test.cpp.o CMakeFiles/ta_test.dir/range1.cpp.o CMakeFiles/ta_test.dir/range.cpp.o CMakeFiles/ta_test.dir/type_traits.cpp.o CMakeFiles/ta_test.dir/tensor.cpp.o CMakeFiles/ta_test.dir/tensor_of_tensor.cpp.o CMakeFiles/ta_test.dir/tensor_tensor_view.cpp.o CMakeFiles/ta_test.dir/tensor_shift_wrapper.cpp.o CMakeFiles/ta_test.dir/tiled_range1.cpp.o CMakeFiles/ta_test.dir/tiled_range.cpp.o CMakeFiles/ta_test.dir/blocked_pmap.cpp.o CMakeFiles/ta_test.dir/round_robin_pmap.cpp.o CMakeFiles/ta_test.dir/hash_pmap.cpp.o CMakeFiles/ta_test.dir/cyclic_pmap.cpp.o CMakeFiles/ta_test.dir/replicated_pmap.cpp.o CMakeFiles/ta_test.dir/dense_shape.cpp.o CMakeFiles/ta_test.dir/sparse_shape.cpp.o CMakeFiles/ta_test.dir/distributed_storage.cpp.o CMakeFiles/ta_test.dir/tensor_impl.cpp.o CMakeFiles/ta_test.dir/array_impl.cpp.o CMakeFiles/ta_test.dir/index_list.cpp.o CMakeFiles/ta_test.dir/bipartite_index_list.cpp.o CMakeFiles/ta_test.dir/dist_array.cpp.o CMakeFiles/ta_test.dir/conversions.cpp.o CMakeFiles/ta_test.dir/eigen.cpp.o CMakeFiles/ta_test.dir/dist_op_dist_cache.cpp.o CMakeFiles/ta_test.dir/dist_op_group.cpp.o CMakeFiles/ta_test.dir/dist_op_communicator.cpp.o CMakeFiles/ta_test.dir/tile_op_noop.cpp.o CMakeFiles/ta_test.dir/tile_op_scal.cpp.o CMakeFiles/ta_test.dir/dist_eval_array_eval.cpp.o CMakeFiles/ta_test.dir/dist_eval_unary_eval.cpp.o CMakeFiles/ta_test.dir/tile_op_add.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_add.cpp.o CMakeFiles/ta_test.dir/tile_op_subt.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_subt.cpp.o CMakeFiles/ta_test.dir/dist_eval_binary_eval.cpp.o CMakeFiles/ta_test.dir/tile_op_mult.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_mult.cpp.o CMakeFiles/ta_test.dir/tile_op_contract_reduce.cpp.o CMakeFiles/ta_test.dir/reduce_task.cpp.o CMakeFiles/ta_test.dir/proc_grid.cpp.o CMakeFiles/ta_test.dir/dist_eval_contraction_eval.cpp.o CMakeFiles/ta_test.dir/expressions.cpp.o CMakeFiles/ta_test.dir/expressions_sparse.cpp.o CMakeFiles/ta_test.dir/expressions_complex.cpp.o CMakeFiles/ta_test.dir/expressions_btas.cpp.o CMakeFiles/ta_test.dir/expressions_mixed.cpp.o CMakeFiles/ta_test.dir/foreach.cpp.o CMakeFiles/ta_test.dir/solvers.cpp.o CMakeFiles/ta_test.dir/initializer_list.cpp.o CMakeFiles/ta_test.dir/diagonal_array.cpp.o CMakeFiles/ta_test.dir/retile.cpp.o CMakeFiles/ta_test.dir/tot_dist_array_part1.cpp.o CMakeFiles/ta_test.dir/tot_dist_array_part2.cpp.o CMakeFiles/ta_test.dir/random.cpp.o CMakeFiles/ta_test.dir/trace.cpp.o CMakeFiles/ta_test.dir/tot_expressions.cpp.o CMakeFiles/ta_test.dir/annotation.cpp.o CMakeFiles/ta_test.dir/contraction_helpers.cpp.o CMakeFiles/ta_test.dir/s_t_t_contract_.cpp.o CMakeFiles/ta_test.dir/t_t_t_contract_.cpp.o CMakeFiles/ta_test.dir/t_s_t_contract_.cpp.o CMakeFiles/ta_test.dir/einsum.cpp.o CMakeFiles/ta_test.dir/linalg.cpp.o CMakeFiles/ta_test.dir/cp.cpp.o CMakeFiles/ta_test.dir/librett.cpp.o CMakeFiles/ta_test.dir/expressions_cuda_um.cpp.o CMakeFiles/ta_test.dir/tensor_um.cpp.o -o ta_test   -L/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/umpire-build/lib  -L/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/librett-build/src  -Wl,-rpath,/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/src:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/madness-build/src/madness/world:/home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/lapackpp-build:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/blaspp-build:/home/dbwy/Software/Boost/gcc/9.4.0/1.80.0/install/lib:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/umpire-build/lib:/usr/local/cuda-11.7/lib64:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/librett-build/src:/home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-libflame/lib/LP64:/home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-blis/lib/LP64 ../src/libtiledarray.so ../_deps/madness-build/src/madness/world/libMADworld.so /home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib/libmpicxx.so /home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib/libmpi.so -lpthread ../_deps/lapackpp-build/liblapackpp.so ../_deps/blaspp-build/libblaspp.so /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so /usr/lib/x86_64-linux-gnu/libpthread.so /home/dbwy/Software/Boost/gcc/9.4.0/1.80.0/install/lib/libboost_serialization.so.1.80.0 -lumpire /usr/local/cuda-11.7/lib64/libcublas.so /usr/local/cuda-11.7/lib64/libcublasLt.so /usr/local/cuda-11.7/lib64/libculibos.a /usr/local/cuda-11.7/lib64/libnvToolsExt.so -lrett /home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-libflame/lib/LP64/libflame.so /home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-blis/lib/LP64/libblis.so -lm 
/usr/bin/ld: CMakeFiles/ta_test.dir/dist_array.cpp.o: in function `array_suite::serialization_by_tile::test_method()':
/home/dbwy/Development/NWChemEx/nwx_update_ta/tiledarray/tests/dist_array.cpp:652: warning: the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp'
/usr/bin/ld: CMakeFiles/ta_test.dir/expressions_cuda_um.cpp.o: undefined reference to symbol 'cudaStreamSynchronize@@libcudart.so.11.0'