grins does not honor LIBMESH_LIBS

[capitalaslash]

when compiling grins, I had an error in linking phase, where openmpi libraries where not found. in fact the linking command was missing -L/usr/lib/openmpi (and adding it to LDFLAGS solved it for me), but it is present in the LIBMESH_LIBS variable in the Makefile. this is true also for runtime paths that libmesh has that are not transported to grins.

[pbauman]

First, thanks for reporting this and thanks for your interest in GRINS! If you're willing, post your interests to grins-users or email me privately so that I make sure we don't interfere with your work. FYI, there's a major overhaul of the input file coming shortly.

To your report, are you trying to specify your own compilation flags at compile time using the --disable-libmesh-flags flags? If not, there may be an issue --- we pull the flags from libmesh-config. Could you confirm that -L/usr/lib/openmpi is or is not present in $LIBMESH_DIR/bin/libmesh-config --libs, where $LIBMESH_DIR is where you've installed libMesh. By default this reports flags for METHOD=opt. You can prefix the call with METHOD=devel or METHOD=dbg if you built GRINS in either of those modes.

I can also have GRINS echo the included libs next to the currently echoed $LIBMESH_DIR and include flags.

[capitalaslash]

I'm just doing my first steps with grins to understand if it fits my needs. I'll come up in the mailing list as soon as I know if I will become a /real/ user... I'll wait and see the update on the input file syntax.

I am not using --disable-libmesh-flag. my configure looks like

configure --prefix=<> --enable-cxx11 --with-libmesh=<> --enable-antioch --with-antioch=<>

the output of libmesh-config --libs | tr ' ' '\n' | grep openmpi is

-L/usr/lib/openmpi
-Wl,-rpath,/usr/lib/openmpi

so it seems to be ok. furthermore, LIBMESH_LIBS inside the grins Makefile stores the correct value, but it is not used in the linking process. my experience with autotools and libtool internals is very limited and i don't know how to further investigate it.

just to clarify, i'm using latest master versions of antioch, libmesh and grins.

[pbauman]

'm just doing my first steps with grins to understand if it fits my needs. I'll come up in the mailing list as soon as I know if I will become a /real/ user...

Sounds good. Please don't hesitate to ask questions.

I'll wait and see the update on the input file syntax.

I've opened the discussion on grins-devel.

my experience with autotools and libtool internals is very limited and i don't know how to further investigate it.

No problem. I'll push a PR quickly to display the libs to help us confirm the problem.

[pbauman]

Ah, OK, I see the problem, I think. Can you confirm whether or not there is a file $LIBMESH_DIR/lib/libmesh_<METHOD>.la? I have this in my install which is why I wouldn't have tripped this. Are you doing make install on your libMesh installation (if not, that would explain the .la file not being there)? Regardless, we want to support the case that is it not there.

(The .la file is a text file that libtool uses to cache dependencies for the library, so if your package uses libtool, you can leverage the .la file to make life easier.)

[capitalaslash]

i am using the installed version of libmesh. i can confirm the presence of $LIBMESH_DIR/lib/libmesh_opt.la, its content is

# libmesh_opt.la - a libtool library file
# Generated by libtool (GNU libtool) 2.4.2
#
# Please DO NOT delete this file!
# It is necessary for linking the library.

# The name that we can dlopen(3).
dlname='libmesh_opt.so.0'

# Names of this library.
library_names='libmesh_opt.so.0.0.0 libmesh_opt.so.0 libmesh_opt.so'

# The name of the static archive.
old_library=''

# Linker flags that can not go in dependency_libs.
inherited_linker_flags=' -fopenmp'

# Libraries that this one depends upon.
dependency_libs=' -L/opt/hdf5-1.8.13/lib /data/aslash/software/libmesh-git/install/lib/libnetcdf.la -lhdf5_hl -lcurl -L/usr/lib -L/usr/lib/vtk-6.1 -L/opt/petsc/arch-linux2-cxx-opt/lib -lhdf5 -lglpk -lvtkIOCore-6.1 -lvtkCommonCore-6.1 -lvtkCommonDataModel-6.1 -lvtkFiltersCore-6.1 -lvtkIOXML-6.1 -lvtkImagingCore-6.1 -lvtkIOImage-6.1 -lvtkImagingMath-6.1 -lpetsc -lHYPRE -lpastix -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lsuperlu_4.3 -lml -lumfpack -lklu -lcholmod -lbtf -lccolamd -lcolamd -lcamd -lamd -lsuitesparseconfig -llapack -lblas -lparmetis -lmetis -lX11 -lptesmumps -lptscotch -lptscotcherr -lscotch -lscotcherr -lssl -lcrypto -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lgfortran -lquadmath -lrt -lz -ldl'

# Names of additional weak libraries provided by this library
weak_library_names=''

# Version information for libmesh_opt.
current=0
age=0
revision=0

# Is this an already installed library?
installed=yes

# Should we warn about portability when linking against -modules?
shouldnotlink=no

# Files to dlopen/dlpreopen
dlopen=''
dlpreopen=''

# Directory that this library needs to be installed in:
libdir='/data/aslash/software/libmesh-git/install/lib'

i see now that the openmpi path is not present in the dependency_libs, so the problem stems from the libtool generated file in libmesh.

[pbauman]

Hmmm, this is strange. How did you configure libMesh? In particular, did you use CXX=mpicxx CC=mpicc when you configured? I wonder if libMesh didn't find MPI (it should report this when you configure)?

[capitalaslash]

this is my libmesh configure script

#! /bin/bash

../configure \
  --prefix=/data/aslash/software/libmesh-git/install \
  \
  --disable-default-comm-world \
  --enable-parmesh \
  --enable-blocked-storage \
  \
  --with-methods="opt dbg" \
  \
  --with-vtk-include=/usr/include/vtk-6.1 \
  --with-vtk-lib=/usr/lib/vtk-6.1 \
  --enable-hdf5 \
  --with-hdf5=/opt/hdf5-1.8.13 \
  --with-tbb=yes \
  $*

[pbauman]

I would try adding to configure CC=/path/to/mpi/mpicc CXX=/path/to/mpi/mpicxx to get to make sure libMesh finds MPI.

[capitalaslash]

i'm pretty sure mpi is enabled. from the script output:

----------------------------------- SUMMARY -----------------------------------

Package version.................... : libmesh-1.0.0-pre

C++ compiler type.................. : gcc4.9
C++ compiler....................... : mpicxx
C compiler......................... : mpicc
Fortran compiler................... : mpif90
Build Methods...................... : opt dbg

CPPFLAGS...(opt)................... : -DNDEBUG
CXXFLAGS...(opt)................... : -std=gnu++11 -O2 -felide-constructors -funroll-loops -fstrict-aliasing -Wdisabled-optimization -fopenmp -std=gnu++11
CFLAGS.....(opt)................... : -O2 -funroll-loops -fstrict-aliasing -fopenmp

CPPFLAGS...(dbg)................... : -DDEBUG -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC
CXXFLAGS...(dbg)................... : -std=gnu++11 -O0 -felide-constructors -g -pedantic -W -Wall -Wextra -Wno-long-long -Wunused -Wpointer-arith -Wformat -Wparentheses -Woverloaded-virtual -fopenmp -std=gnu++11
CFLAGS.....(dbg)................... : -g -Wimplicit -fopenmp

Install dir........................ : /data/aslash/software/libmesh-git/install
Build user......................... : aslash
Build host......................... : utfissm-cervone
Build architecture................. : x86_64-unknown-linux-gnu
Git revision....................... : a0f5691e78b6baf56c9828d796c46a257a85dcab

Library Features:
  library warnings................. : yes
  adaptive mesh refinement......... : yes
  blocked matrix/vector storage.... : yes
  complex variables................ : no
  example suite.................... : yes
  ghosted vectors.................. : yes
  high-order shape functions....... : yes
  unique-id support................ : no
  id size (boundaries)............. : 2 bytes
  id size (dofs)................... : 4 bytes
  id size (processors)............. : 2 bytes
  id size (subdomains)............. : 2 bytes
  infinite elements................ : no
  Dirichlet constraints............ : yes
  node constraints................. : no
  parallel mesh.................... : yes
  performance logging.............. : no
  periodic boundary conditions..... : yes
  reference counting............... : yes
  shape function 2nd derivatives... : yes
  stack trace files................ : no
  track node valence............... : yes
  variational smoother............. : yes
  xdr binary I/O................... : yes

Optional Packages:
  boost............................ : yes
  cppunit.......................... : yes
  eigen............................ : yes
  exodus........................... : yes
     version....................... : v5.22
  fparser.......................... : yes
     build from version............ : release
  glpk............................. : yes
  gmv.............................. : yes
  gzstream......................... : yes
  hdf5............................. : yes
  laspack.......................... : no
  libhilbert....................... : yes
  metis............................ : yes
  mpi.............................. : yes
  nanoflann........................ : yes
  nemesis.......................... : yes
     version....................... : v5.22
  netcdf........................... : yes
     version....................... : 4
  openmp........................... : yes
  parmetis......................... : yes
  petsc............................ : yes
     version....................... : 3.5.2
  slepc............................ : no
  pthreads......................... : yes
  qhull............................ : yes
  sfcurves......................... : no
  slepc............................ : no
  tbb.............................. : no
  c++ threads...................... : yes
     flavor........................ : std::thread
  c++ rtti ........................ : yes
  tecio............................ : yes
  tecplot...(vendor binaries)...... : no
  tetgen........................... : yes
  triangle......................... : no
  trilinos......................... : yes
     AztecOO....................... : yes
     NOX........................... : yes
     ML............................ : yes
     Tpetra........................ : yes
     DTK........................... : no
  vtk.............................. : yes
     version....................... : 6.1.0

  libmesh_optional_INCLUDES........ : -I/opt/hdf5-1.8.13/include -I/usr/include -I/usr/include/eigen3 -I/usr/include/vtk-6.1 -I/opt/petsc/arch-linux2-cxx-opt/include -I/opt/petsc/arch-linux2-cxx-opt//include

  libmesh_optional_LIBS............ : -L/opt/hdf5-1.8.13/lib -lhdf5 -Wl,-rpath,/opt/hdf5-1.8.13/lib -L/usr/lib -lglpk -L/usr/lib/vtk-6.1 -lvtkIOCore-6.1 -lvtkCommonCore-6.1 -lvtkCommonDataModel-6.1 -lvtkFiltersCore-6.1 -lvtkIOXML-6.1 -lvtkImagingCore-6.1 -lvtkIOImage-6.1 -lvtkImagingMath-6.1 -lmsqutil -lmesquite -lpytrilinos -lpiro -lstokhos_sacado -lstokhos -lmoochothyra -lmoocho -lrythmos -lmoertel -llocathyra -llocaepetra -llocalapack -lloca -lnoxepetra -lnoxlapack -lnox -lphalanx -lintrepid -lteko -lfei_trilinos -lfei_base -lstratimikos -lstratimikosbelos -lstratimikosaztecoo -lstratimikosamesos -lstratimikosml -lstratimikosifpack -lifpack2-adapters -lifpack2 -lanasazitpetra -lModeLaplace -lanasaziepetra -lanasazi -lbelostpetra -lbelosepetra -lbelos -lkomplex -lifpack -lpamgen_extras -lpamgen -lamesos -lgaleri-xpetra -lgaleri -laztecoo -ldpliris -lisorropia -loptipack -lthyratpetra -lthyraepetraext -lthyraepetra -lthyracore -lxpetra-sup -lxpetra-ext -lxpetra -lepetraext -ltpetraext -ltpetrainout -ltpetra -ltriutils -lglobipack -lshards -lzoltan -lepetra -lsacado -lkokkosdisttsqr -lkokkosnodetsqr -lkokkoslinalg -lkokkosnodeapi -lkokkos -lkokkosTPL_unused_dummy -lgtest -lrtop -ltpi -lteuchosremainder -lteuchosnumerics -lteuchoscomm -lteuchosparameterlist -lteuchoscore /usr/lib64/liblapack.so /usr/lib64/libblas.so -L/opt/petsc/arch-linux2-cxx-opt/lib -lpetsc -Wl,-rpath,/opt/petsc/arch-linux2-cxx-opt/lib -lHYPRE -lpastix -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lsuperlu_4.3 -Wl,-rpath,/usr/lib -lml -lumfpack -lklu -lcholmod -lbtf -lccolamd -lcolamd -lcamd -lamd -lsuitesparseconfig -llapack -lblas -lparmetis -lmetis -lX11 -lptesmumps -lptscotch -lptscotcherr -lscotch -lscotcherr -lssl -lcrypto -L/usr/lib/openmpi -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.2 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lgfortran -lquadmath -lmpi_cxx -lstdc++ -lrt -lm -lz -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.2 -Wl,-rpath,/usr/lib/openmpi -lmpi -lgcc_s -lpthread -ldl

-------------------------------------------------------------------------------

the openmpi folder is included in the libmesh_optional_LIBS variable, but it does not reach the libtool .la file.

[pbauman]

Indeed, it did find MPI. I'll look into this. In the mean time, supplying LDFLAGS with GRINS configure got you working?

[pbauman]

Could you supply the output of libtool --version please. Thanks again.

[pbauman]

Could you supply the output of libtool --version please.

Actually, nevermind, configure generates the libtool script based on the 2.4.2 version used to generate the configure. This wouldn't change unless you ran bootstrap with libMesh.

[capitalaslash]

LDFLAGS got me working. i did not run bootstrap in libmesh, but for grins i did, my version is 2.4.5