Trilinos installation

[MarkusMehnert]

Hi,

I am currently trying install the dealii suite but during the installtion of Trilinos I get an error message. The patching and the cmake command seem to work but during "make VERBOSE=1" the installation stops with an error. The only error messages I find in the log file 02.make all state: "collect2: error: ld returned 1 exit status".

The last few lines in the log file state: "[ 64%] Built target ctrilinos make[1]: Verzeichnis »/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build« wird verlassen make: *\ [all] Fehler 2"

Thank you for your help, Regards, Markus

[davydden]

pls run brew gist-logs trilinos and post the link here.

[davydden]

you can also create a gist yourself: https://gist.github.com just upload 02.make

[MarkusMehnert]

https://gist.github.com/a782e5eb7bcaabb9ba89

[davydden]

link does not work, you are sure it's not private?

[MarkusMehnert]

ok, here is the link, I think it should work

https://gist.github.com/a782e5eb7bcaabb9ba89

[davydden]

there are several missing symbols errors alike:

cd /tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/amesos/example && /usr/bin/cmake -E cmake_link_script CMakeFiles/Amesos_example_AmesosFactory.dir/link.txt --verbose=1
/usr/bin/mpicxx     -Os -w -pipe -march=core2  -std=c++11 -O3   -L/calculate/iwtm80/.linuxbrew/lib -Wl,-rpath,/calculate/iwtm80/.linuxbrew/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.8 -lgfortran CMakeFiles/Amesos_example_AmesosFactory.dir/example_AmesosFactory.cpp.o  -o Amesos_example_AmesosFactory.exe -rdynamic ../src/libamesos.so.12.2.1 ../../galeri/src-xpetra/libgaleri-xpetra.so.12.2.1 ../../galeri/src/libgaleri.so.12.2.1 ../../triutils/src/libtriutils.so.12.2.1 /calculate/iwtm80/.linuxbrew/opt/mumps/lib/libdmumps.so /calculate/iwtm80/.linuxbrew/opt/mumps/lib/libpord.so /calculate/iwtm80/.linuxbrew/opt/mumps/lib/libmumps_common.so /calculate/iwtm80/.linuxbrew/lib/libscalapack.so /calculate/iwtm80/.linuxbrew/lib/libsuperlu_dist.a /calculate/iwtm80/.linuxbrew/opt/parmetis/lib/libparmetis.so /calculate/iwtm80/.linuxbrew/opt/metis/lib/libmetis.so ../../xpetra/sup/libxpetra-sup.so.12.2.1 ../../xpetra/ext/libxpetra-ext.so.12.2.1 ../../xpetra/src/libxpetra.so.12.2.1 ../../epetraext/src/libepetraext.so.12.2.1 ../../triutils/src/libtriutils.so.12.2.1 /calculate/iwtm80/.linuxbrew/lib/libHYPRE.a /calculate/iwtm80/.linuxbrew/lib/libhdf5.so /calculate/iwtm80/.linuxbrew/lib/libsz.so /calculate/iwtm80/.linuxbrew/lib/libz.so -ldl -lm ../../tpetra/core/ext/libtpetraext.so.12.2.1 ../../tpetra/core/inout/libtpetrainout.so.12.2.1 ../../tpetra/core/src/libtpetra.so.12.2.1 ../../epetra/src/libepetra.so.12.2.1 ../../kokkos/algorithms/src/libkokkosalgorithms.so.12.2.1 ../../tpetra/tsqr/src/libkokkostsqr.so.12.2.1 ../../tpetra/kernels/src/libtpetrakernels.so.12.2.1 ../../kokkos/containers/src/libkokkoscontainers.so.12.2.1 ../../tpetra/classic/LinAlg/libtpetraclassiclinalg.so.12.2.1 ../../tpetra/classic/NodeAPI/libtpetraclassicnodeapi.so.12.2.1 ../../tpetra/classic/src/libtpetraclassic.so.12.2.1 ../../teuchos/kokkoscomm/src/libteuchoskokkoscomm.so.12.2.1 ../../teuchos/kokkoscompat/src/libteuchoskokkoscompat.so.12.2.1 ../../teuchos/remainder/src/libteuchosremainder.so.12.2.1 ../../teuchos/numerics/src/libteuchosnumerics.so.12.2.1 -lblas -llapack -lblas -llapack ../../teuchos/comm/src/libteuchoscomm.so.12.2.1 ../../teuchos/parameterlist/src/libteuchosparameterlist.so.12.2.1 ../../teuchos/core/src/libteuchoscore.so.12.2.1 ../../kokkos/core/src/libkokkoscore.so.12.2.1 ../../ThreadPool/src/libtpi.so.12.2.1 -Wl,-rpath,/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/amesos/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/galeri/src-xpetra:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/galeri/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/triutils/src:/calculate/iwtm80/.linuxbrew/opt/mumps/lib:/calculate/iwtm80/.linuxbrew/lib:/calculate/iwtm80/.linuxbrew/opt/parmetis/lib:/calculate/iwtm80/.linuxbrew/opt/metis/lib:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/xpetra/sup:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/xpetra/ext:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/xpetra/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/epetraext/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/core/ext:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/core/inout:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/core/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/epetra/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/kokkos/algorithms/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/tsqr/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/kernels/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/kokkos/containers/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/classic/LinAlg:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/classic/NodeAPI:/tmp/
trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/tpetra/classic/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/kokkoscomm/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/kokkoscompat/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/remainder/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/numerics/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/comm/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/parameterlist/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/teuchos/core/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/kokkos/core/src:/tmp/trilinos20151029-6804-15l1hkr/trilinos-12.2.1-Source/build/packages/ThreadPool/src 
/calculate/iwtm80/.linuxbrew/opt/mumps/lib/libmumps_common.so: Nicht definierter Verweis auf `firstPostorder'
/calculate/iwtm80/.linuxbrew/opt/mumps/lib/libmumps_common.so: Nicht definierter Verweis auf `nextPostorder'
/calculate/iwtm80/.linuxbrew/opt/mumps/lib/libmumps_common.so: Nicht definierter Verweis auf `SPACE_ordering'
/calculate/iwtm80/.linuxbrew/opt/mumps/lib/libmumps_common.so: Nicht definierter Verweis auf `freeElimTree'
collect2: error: ld returned 1 exit status

if you are not building on a cluster, try brew test mumps to see if mumps works.

[davydden]

also check that pord library is build as a part of mumps.

[jppelteret]

HI Markus,

So, as Denis was saying, do you see libpord.so when executing

ls $(brew --prefix)/opt/mumps/lib

and, if so, does this call return anything sensible

nm -g $(brew --prefix)/opt/mumps/lib/libpord.so | grep firstPostorder

?

J-P

[MarkusMehnert]

Hi J-P,

I can see libpord.so in the mumps library but when runnig the second command it doesn't return anything.

Markus

[davydden]

It seems your mumps installation is brocken. Note that mums depends on metis and parmetis, so i would first try to make both work within linuxbrew (do not self-compile parmetis).

[MarkusMehnert]

Hi Denis,

So I thought I just start the installation from scratch but again I get an error during the installation of parmetis (the log file https://gist.github.com/ed013ec289f4d1dc6e54).

[davydden]

undefined reference to symbol 'pow@@GLIBC_2.2.5'

I saw this already... Need to go now, will post later...

[jppelteret]

I think that I encountered this issue on Ubuntu and it may have something to do with GCC 4.8 (and 4.9; see my last comment in here). I had briefly spoken to Christian about this yesterday. Perhaps he can upgrade your GCC to 5.x. On Ubuntu 14.04, one does this as follows:

### Upgrade GCC to 5.x series
sudo apt-get install linux-headers-$(uname -r)
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install g++-5 gcc-5 gfortran-5
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60  --slave /usr/bin/g++ g++ /usr/bin/g++-5  --slave /usr/bin/gfortran gfortran /usr/bin/gfortran-5

# Test
gfortran -v && g++ -v && gcc -v

I'll update the readme file accordingly.

[jppelteret]

I've managed to run into the same problem as @MarkusMehnert, even with an updated GCC, on a fresh installation (boo :-( ) so I'll see if I can work out whats going on here.

[jppelteret]

@davydden Do you think that this is perhaps due to a broken Trilinos formula? Has something changed between the time when I was checking your BLAS library detection and now? I see that the version is the same (Trilinos 12.2.1). According to this post, you see these sorts of issues when libpord is not linked in properly.

[davydden]

no, i did not change anything since then. If you still have it on a virtual machine you can diff the two, unfortunately i re-created the repo as originally i forked from your old dealii-suite.

In either case the problem is NOT with Trilinos, but with mumps as the resulting librareis don't have those symbols. Why this happens - i do not know. I can only speculate that this can be related to issues with parametis and the fact that @MarkusMehnert had to build it himself.

[davydden]

more likely it's related to the order of libraries in the linking line: linker behaviour.

[jppelteret]

Ok well, you've given me some things think about and explore next week.

Just to clarify, I reproduced problem in an Ubuntu VM when using this repo and the build instructions we've listed in the README (no self-compiled packages), which is not dissimilar to the method that I had successfully used in those BLAS tests.

[jppelteret]

My build log for Trilinos

[jppelteret]

The make log

[davydden]

wait a second, so you built parmetis without any issues using Linuxbrew?

[davydden]

a whild guess, how about changing the sequence dmumps;pord;mumps_common in Trilinos to mumps_common;dmumps;pord?

p.s. that's only assuming that the missing symbols are there in libpord.

[davydden]

the sequence of libs in your error log is indeed wrong (follows dmumps;pord;mumps_common):

/home/jp/.linuxbrew/opt/mumps/lib/libdmumps.so /home/jp/.linuxbrew/opt/mumps/lib/libpord.so /home/jp/.linuxbrew/opt/mumps/lib/libmumps_common.so 

[jppelteret]

Ok I'm giving your suggestion a go. Hopefully it won't take too long to get past that build point.

As you your earlier question, yes I built parmetis without a problem. As I exmplained to Christian yesterday, one needs to upgrade GCC to 5.x to fix this problem. Don't know what causes it, but thats a fix.

[davydden]

at least parmetis is good!

[jppelteret]

That fixed one problem and caused another. We'll pick this up on Monday...

[davydden]

this suggests that the correct order is dmumps;mumps_common;pord

[jppelteret]

Beat you to it - I'm busy rebuilding with that ordering :-)

[davydden]

:-)

p.s. those missing simbols sit in mumps_common:

$ nm -gU /usr/local/opt/mumps/lib/libmumps_common.dylib | grep mumps_irealloc
0000000000026ff2 T _mumps_irealloc_

[jppelteret]

So its gotten further than before but it'll take quite some time to finish building. Hopefully I'll arrive on Monday to a successful build. Will keep you posted...

[jppelteret]

@MarkusMehnert We've made a number of fixes to the formulae that should hopefully fix all of the problems that you've encountered. We found where the problem was with both parmetis and trilinos (well, mumps really) and have corrected those. One now doesn't need to upgrade GCC to version 5.1 either. We've also written an installation script for Ubuntu which will automate the whole process for you.

If you haven't successfully built deal.II yet, then can you please delete your current Linuxbrew directory and start from scratch either using the new build instructions on the front page / in the README or via the installation script that is also mentioned there. You'll need the administrator password once unless you edit the script to remove the affected parts. I think that you can safely do this since the necessary packages should have been installed already.

[MarkusMehnert]

Works like a charm.

Thank you guys

[jppelteret]

Awesome, glad to know that its sorted now :-)

[davydden]

cool! i close the issue then.