Test Linkage Error

[marshallrdavey]

This happens on master and at #166 (even though it was working then when I submitted it). This gist of the issue is that when I try to compile the tests, I get the following error.

/usr/bin/ld: ../libfiddle2d.so: undefined reference to symbol 'MPI_Bcast'
/not_backed_up/heart/programs/petsc/3.10.5/mpich/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
tests/CMakeFiles/tests-base_qwv_family_01.dir/build.make:233: recipe for target 'tests/base/qwv_family_01' failed
make[3]: *** [tests/base/qwv_family_01] Error 1
CMakeFiles/Makefile2:1388: recipe for target 'tests/CMakeFiles/tests-base_qwv_family_01.dir/all' failed
make[2]: *** [tests/CMakeFiles/tests-base_qwv_family_01.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
/usr/bin/ld: ../libfiddle2d.so: undefined reference to symbol 'MPI_Bcast'
/not_backed_up/heart/programs/petsc/3.10.5/mpich/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
tests/CMakeFiles/tests-base_nonintersecting_sphere_01_2d.dir/build.make:233: recipe for target 'tests/base/nonintersecting_sphere_01_2d' failed
make[3]: *** [tests/base/nonintersecting_sphere_01_2d] Error 1
CMakeFiles/Makefile2:1362: recipe for target 'tests/CMakeFiles/tests-base_nonintersecting_sphere_01_2d.dir/all' failed
make[2]: *** [tests/CMakeFiles/tests-base_nonintersecting_sphere_01_2d.dir/all] Error 2
/usr/bin/ld: ../libfiddle2d.so: undefined reference to symbol 'MPI_Bcast'
/not_backed_up/heart/programs/petsc/3.10.5/mpich/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
tests/CMakeFiles/tests-interaction_dlm_01.dir/build.make:233: recipe for target 'tests/interaction/dlm_01' failed
make[3]: *** [tests/interaction/dlm_01] Error 1
CMakeFiles/Makefile2:1163: recipe for target 'tests/CMakeFiles/tests-interaction_dlm_01.dir/all' failed
make[2]: *** [tests/CMakeFiles/tests-interaction_dlm_01.dir/all] Error 2
/usr/bin/ld: ../libfiddle2d.so: undefined reference to symbol 'MPI_Bcast'
/not_backed_up/heart/programs/petsc/3.10.5/mpich/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
tests/CMakeFiles/tests-base_base64.dir/build.make:233: recipe for target 'tests/base/base64' failed
make[3]: *** [tests/base/base64] Error 1
CMakeFiles/Makefile2:1137: recipe for target 'tests/CMakeFiles/tests-base_base64.dir/all' failed
make[2]: *** [tests/CMakeFiles/tests-base_base64.dir/all] Error 2
/usr/bin/ld: ../libfiddle2d.so: undefined reference to symbol 'MPI_Bcast'
/not_backed_up/heart/programs/petsc/3.10.5/mpich/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
tests/CMakeFiles/tests-base_initial_guess.dir/build.make:233: recipe for target 'tests/base/initial_guess' failed
make[3]: *** [tests/base/initial_guess] Error 1
CMakeFiles/Makefile2:306: recipe for target 'tests/CMakeFiles/tests-base_initial_guess.dir/all' failed
make[2]: *** [tests/CMakeFiles/tests-base_initial_guess.dir/all] Error 2

I previously saw this error in IBAMR/fch-deal issue 55 which I resolved by moving some functions from a source file to a header. I honestly have no idea where to start with this issue. I was using my Trilinos build of fiddle for the last PR I submitted, so I don't think I have touched anything upstream of fiddle since it last compiled and ran the tests.

[drwells]

That's an odd error because it implies that we are no longer correctly linking against MPI. Can you use the provided MPI compiler wrappers to build a basic "hello world MPI" program?

[marshallrdavey]

How? Just build hello world with mpic++?

[drwells]

Essentially yes, but include some basic MPI calls (like a stripped-down version of the classic ping-pong example) to make sure we can correctly link against MPI.

[marshallrdavey]

Built at ran this example with 4 processes fine. I had to change the header from to because it couldn't find cout/endl.

[drwells]

That's a good start. Can you successfully compile and link deal.II on top of that version of PETSc with that same version of MPI?

[marshallrdavey]

That's what I've been building deal.II with for years, but I can rebuild.

[marshallrdavey]

Built and installed dealii just fine.

[marshallrdavey]

When I was looking previously for solutions, I found this line in IBAMR but it seemed completely unrelated. https://github.com/IBAMR/IBAMR/blob/5d56fcce3317f6aadd690a21683be39c129efb2f/ibtk/include/ibtk/private/IBTK_MPI-inl.h#L108

[drwells]

Perhaps the problem is with IBAMR. Can you check by compiling some examples or tests with deal.II and then the same for IBAMR?

[marshallrdavey]

All of the deal.ii steps compiled properly, and all of the IBAMR tests and examples compile properly. I'm using some compiler flags in fiddle, but I haven't changed them in years.

[drwells]

Interesting - I don't know what would cause this problem since we inherit the MPI environment from IBAMR.

Do you use MPI compiler wrappers for both IBAMR and fiddle?

[marshallrdavey]

I specify -DMPI_ROOT=$MPI for IBAMR and in dealii I point to the specific compilers.

     -DCMAKE_C_COMPILER=$MPI/mpicc \
     -DCMAKE_CXX_COMPILER=$MPI/mpicxx \
     -DCMAKE_Fortran_COMPILER=$MPI/mpif90 \

[marshallrdavey]

I'm going to see if I can isolate the offending tests.

[marshallrdavey]

Here are the tests that fail to compile. The rest compile and pass.

SETUP(base qgauss_family_01.cc fiddle3d)
SETUP(base qgauss_family_02.cc fiddle3d)
SETUP(base qwv_family_01.cc fiddle2d)
SETUP(base initial_guess.cc fiddle2d)
SETUP(base base64.cc fiddle2d)
SETUP_2D(base nonintersecting_sphere_01.cc)
SETUP_3D(base nonintersecting_sphere_01.cc)
SETUP_2D(grid surface_tria_01.cc)
SETUP_2D(grid exodus.cc)
SETUP_3D(grid extract_nodeset_01.cc)
SETUP(grid overlap_tria_01.cc fiddle2d)
SETUP(interaction dlm_01.cc fiddle2d)

[marshallrdavey]

Is there a way to set the normal C and CXX compilers to mpicc and mpicxx instead, or should this not make a difference? When I look at the CMakeFiles I see that it is still linking to /usr/bin/cc and /usr/bin/c++.

[marshallrdavey]

IBAMR is compiling with the native g compilers and the mpi compilers. I might try to force it to use the mpi compilers for everything as in trilinos and dealii.

[marshallrdavey]

@drwells I figured it out. I had to explicitly specify to CMake that I wanted to use the MPI compilers for everything (I did this for IBAMR and fiddle, and it ultimately worked when I did this for fiddle). I can make a PR for the README to suggest as much if you think that would be useful. I'm just glad everything is working again.

[drwells]

Good to hear.

What you did should have worked because IBAMR always links against MPI (it is part of the public link interface). Something must be wrong with how IBAMR sets up MPI as an exported target. I'll take a look this morning.

[marshallrdavey]

fiddle found MPI from IBAMR, it was just linking to the gnu compilers on my machine as well, at least for the tests. I may be botching my explanation of this.

[marshallrdavey]

Fixed by an IBAMR patch