Build Error with MPI and C++ Examples Enabled

[Steven-Roberts]

The build command cmake -DENABLE_MPI=ON -DEXAMPLES_ENABLE_CXX=ON .. causes the following build error for me

CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o: In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)':                                        
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/op_inln.h:121: undefined reference to `ompi_mpi_cxx_op_intercept'                                                                         
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o: In function `MPI::Intracomm::Clone() const':                                                                                       
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'                                                                           
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o: In function `MPI::Graphcomm::Clone() const':                                                                                       
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()'                                                                                
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o: In function `MPI::Cartcomm::Sub(bool const*) const':                                                                               
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()'                                                                                
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o: In function `MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const':                                               
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()'                                                                                
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o: In function `MPI::Cartcomm::Clone() const':                                                                                        
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()'                                                                                
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o:/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: more undefined references to `MPI::Comm::Comm()' follow       
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o:(.rodata._ZTVN3MPI8DatatypeE[_ZTVN3MPI8DatatypeE]+0x78): undefined reference to `MPI::Datatype::Free()'                             
CMakeFiles/ark_analytic_sys.dir/ark_analytic_sys.cpp.o:(.rodata._ZTVN3MPI3WinE[_ZTVN3MPI3WinE]+0x48): undefined reference to `MPI::Win::Free()'                                            
collect2: error: ld returned 1 exit status
make[2]: *** [examples/arkode/CXX_serial/CMakeFiles/ark_analytic_sys.dir/build.make:102: examples/arkode/CXX_serial/ark_analytic_sys] Error 1                                              
make[1]: *** [CMakeFiles/Makefile2:6810: examples/arkode/CXX_serial/CMakeFiles/ark_analytic_sys.dir/all] Error 2                                                                           
make: *** [Makefile:146: all] Error 2

I'm using gcc 8.5 and Open MPI 4.1.1. With mpich 4.1.1 it compiles without error.

[balos1]

Can you try other versions of OpenMPI, such as 4.1.2? Seems like it could be an OpenMPI problem. I am using OpenMPI 5.x with gcc 8.5 and have not had an issue.

[Steven-Roberts]

I tried OpenMPI 4.1.2 and GCC 13.2 (via spack instead of system packages) and still get the same issue. Interestingly when building on quartz with OpenMPI 4.1.2 it compiles fine. I'm not sure what to make of that.

The CXX_serial/ark_analytic_sys example is compiled with g++ (and not mpicxx), which I think is fine, but somehow when linking, the MPI functions are needed...

[balos1]

What CMake version are you using? As of SUNDIALS 7, when you build SUNDIALS with MPI enabled all of SUNDIALS requires MPI because of the new SUNComm (so that is why the MPI symbols are required). However, CMake should be adding the MPI::MPI_CXX target in and thus the MPI libraries should be getting linked to (even though youre compiling with g++). In older CMake versions it did a bad job of propagating things related to MPI correctly.

[Steven-Roberts]

What CMake version are you using?

3.26.5

[Steven-Roberts]

Fixed in https://github.com/LLNL/sundials/pull/469