How do we build a serial application using ADIOS2 combined MPI/no-MPI library?

[pnorbert]

I am trying to build visit with the combined library, but it does not compile. It is using g++ without any MPI links. Including adios2.h, however, leads to including mpi.h, which is not found. Do we have a serial code test that shows how to build it with adios?

[ 93%] Building CXX object databases/ADIOS2/CMakeFiles/MADIOS2Database.dir/avtADIOS2FileFormat.C.o
cd /work/adios/visit3/build/databases/ADIOS2 && 
/usr/bin/g++   ...   -c /work/adios/visit3/visit/src/databases/ADIOS2/avtADIOS2FileFormat.C
In file included from /opt/adios2/include/adios2/cxx11/ADIOS.h:18:0,
                 from /opt/adios2/include/adios2.h:18,
                 from /work/adios/visit3/visit/src/databases/ADIOS2/avtADIOS2BaseFileFormat.h:49,
                 from /work/adios/visit3/visit/src/databases/ADIOS2/avtADIOS2FileFormat.C:48:
/opt/adios2/include/adios2/cxx11/IO.h:20:17: fatal error: mpi.h: No such file or directory

Adding -UADIOS2_HAVE_MPI to the compile line did not help with this issue.

[germasch]

To make this work, one will have to compile with mpicxx, or add the correct include path. One also needs to link with the MPI library (or again use mpicxx for linking, at least with static lib). That's the approach advocated by @ax3l and the one that got chosen. (Note that there isn't really any way around at least the linking side of things, unless you build two separate libraries or try do dynamic loading of MPI on demand right.)

Now the question is why this isn't working automatically, since it appears that you're using cmake, and it should pull in MPI. I hope @chuckatkins can chime on this, as he really knows how this is supposed to work, but here are my observations:

• to use ADIOS2 from a cmake project, one should presumably use find_package. I don't know if a FindADIOS2.cmake exists, I'm not aware of any, so the way this would currently work is based on adios2-config.cmake, which gets installed together with adios2. In the ADIOS2_HAVE_MPI case, which we are talking about here, this creates an adios2::adios2 target that adds an interface link dependency on MPI:

INTERFACE_LINK_LIBRARIES "MPI::MPI_C"

So If you link your code with adios2::adios2, this should pull in the MPI library both in terms of header include paths as well as linking with the library, ie., you shouldn't get that error.

So I guess the question then is, how are you building visit? (in particular how does it handle linking with adios2)

A somewhat related issue is that if you use a regular Makefile and adios2-config to get the include / lib flags, it will not link with MPI automatically. I presume the assumption is that in this case you really have to use mpicxx/mpicc, @chuckatkins?

[ax3l]

With 2.4.0, I am also still experiencing problems to import adios2 in python if it was build with MPI support and parallel HDF5:

~~ImportError: lib/python3.7/site-packages/../../libadios2.so.2:~~
~~undefined symbol: H5Pset_dxpl_mpio~~

Probably we just need to forward a bit more linkage here, as the problem above.

Update: forget what I wrote, just a setting error with deps in a package manager.

[pnorbert]

This should not happen if your LD_LIBRARY_PATH points to the parallel HDF5 library, not the serial one.

On Mon, Jul 1, 2019 at 12:24 PM Axel Huebl notifications@github.com wrote:

With 2.4.0, I am also still experiencing problems to import adios2 in python if it was build with MPI support and parallel HDF5:

ImportError: lib/python3.7/site-packages/../../libadios2.so.2: undefined symbol: H5Pset_dxpl_mpio

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ornladios/ADIOS2/issues/1558?email_source=notifications&email_token=AAYYYLLJPWFTRRRNK53QPPLP5IVUNA5CNFSM4H343IIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY6U7UI#issuecomment-507334609, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYYYLNLMAAPWRZPVGQPPFLP5IVUNANCNFSM4H343IIA .

[ax3l]

@pnorbert yes, sorry I just update my comment online. I mixed compile-time and runtime HDF5 variants in a package manager.