search

easybuilders / easybuild-easyconfigs

A collection of easyconfig files that describe which software to build using which build options with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
378 stars 701 forks source link

openmpi-1.10.2 build failure #5898

Open verdurin opened 6 years ago

verdurin commented 6 years ago

Saw this error when testing a PR:

make[2]: Leaving directory `/dev/shm/OpenMPI/1.10.2/GCC-4.9.3-2.25/openmpi-1.10.2/ompi/mca/vprotocol'
Making all in mca/common/verbs_usnic
make[2]: Entering directory `/dev/shm/OpenMPI/1.10.2/GCC-4.9.3-2.25/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
  CC       libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
common_verbs_usnic_fake.c:72:5: error: unknown field 'ops' specified in initializer
     .ops = {
     ^
common_verbs_usnic_fake.c:73:9: error: unknown field 'alloc_context' specified in initializer
         .alloc_context = fake_alloc_context,
         ^
common_verbs_usnic_fake.c:74:9: error: unknown field 'free_context' specified in initializer
         .free_context = fake_free_context
         ^
common_verbs_usnic_fake.c: In function 'fake_driver_init':
common_verbs_usnic_fake.c:89:5: warning: implicit declaration of function 'ibv_read_sysfs_file' [-Wimplicit-function-declaration]
     if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
     ^
common_verbs_usnic_fake.c: In function 'ompi_common_verbs_usnic_register_fake_drivers':
common_verbs_usnic_fake.c:133:9: warning: implicit declaration of function 'ibv_register_driver' [-Wimplicit-function-declaration]
         ibv_register_driver("usnic_verbs", fake_driver_init);
         ^
make[2]: *** [libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo] Error 1
make[2]: Leaving directory `/dev/shm/OpenMPI/1.10.2/GCC-4.9.3-2.25/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/dev/shm/OpenMPI/1.10.2/GCC-4.9.3-2.25/openmpi-1.10.2/ompi'
make: *** [all-recursive] Error 1
 (at easybuild/software/EasyBuild/3.5.1/lib/python2.7/site-packages/easybuild_framework-3.5.1-py2.7.egg/easybuild/tools/run.py:481 in parse_cmd_output)
verdurin commented 6 years ago

This was when trying to build the foss-2016a toolchain.

verdurin commented 6 years ago

Build host is running CentOS 7.4, fully updated.

FokkeDijkstra commented 6 years ago

I ran into the same issue. I think the issue is caused by libibverbs-devel being superseded by rdma-core-devel. Apparently this transition has changed something in the underlying libraries. I fixed it by installing the libibverbs libraries from Mellanox, but I am not sure if that will work out for you.

oneilsh commented 6 years ago

Bummer, this one is hanging me up. I'm trying to start playing with EasyBuild at our center, but we've recently updated everything to Centos 7.4 (not using any infiniband). @FokkeDijkstra, did you get the Mellanox provided libs via this guide? http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Installation_Guide.pdf

boegel commented 6 years ago

The way forward would be to try and avoid using old compiler toolchains... Why not use a more recent version of foss, cfr. http://easybuild.readthedocs.io/en/latest/Common-toolchains.html#overview-of-common-toolchains ?

oneilsh commented 6 years ago

That sounds excellent. I still have a lot of learning about EasyBuild to do, so my apologies if this is something that I missed in the docs... my first test software suite was going to be HMMER, so I did an eb -S hmmer, and saw the listing:

  * $CFGS1/h/HMMER/HMMER-3.0-ictce-5.3.0.eb
  * $CFGS1/h/HMMER/HMMER-3.1b1-goolf-1.4.10.eb
  * $CFGS1/h/HMMER/HMMER-3.1b1-ictce-5.3.0.eb
  * $CFGS1/h/HMMER/HMMER-3.1b1-ictce-6.2.5.eb
  * $CFGS1/h/HMMER/HMMER-3.1b2-foss-2016a.eb
  * $CFGS1/h/HMMER/HMMER-3.1b2-intel-2015a.eb
  * $CFGS1/h/HMMER/HMMER-3.1b2-intel-2017a.eb
  * $CFGS1/h/HMMER/hmmer-3.1b1_link-LIBS-utests.patch

I'd prefer foss to intel, but I only see foss-2016a as an available toolchain above. Would this be a place to --try-toolchain=foss,2018a or similar? Or should I just try with the intel-2017a chain (though I don't think we have any licenses for it...)

boegel commented 6 years ago

In this case, I would definitely recommend trying with eb HMMER-3.1b2-intel-2017a.eb --try-toolchain=foss,2018a --robot. HMMER doesn't seem to have any dependencies, and I can't spot anything specific to Intel compilers in that easyconfig file either, so chances are pretty good it'll work without too much hassle.

Bonus points 🍪 if you contribute back the resulting easyconfig file (cfr. http://easybuild.readthedocs.io/en/latest/Contributing.html#contributing-easyconfigs).

oneilsh commented 6 years ago

Ok, it's building-- I'll go for those bonus points if I manage to cross the finish line. Thanks!

oneilsh commented 6 years ago

Yay, it worked! And I think I even submitted the PR properly: https://github.com/easybuilders/easybuild-easyconfigs/pull/6276 (Also, it's awesome that contributing back is so easy with EasyBuild's GitHub integration.)

My next test was with bowtie2, but I think I might have run across a different error. Is this something that I should report as an issue, or should I try with other --try-toolchain options?

Here's my result of eb Bowtie2-2.3.2-intel-2017a.eb -Dr --try-toolchain=foss,2018a

 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/h/help2man/help2man-1.47.4.eb (module: help2man/1.47.4)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/f/flex/flex-2.6.3.eb (module: flex/2.6.3)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/b/binutils/binutils-2.28.eb (module: binutils/2.28)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/g/GCCcore/GCCcore-6.4.0.eb (module: GCCcore/6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/m/M4/M4-1.4.18-GCCcore-6.4.0.eb (module: M4/1.4.18-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/z/zlib/zlib-1.2.11-GCCcore-6.4.0.eb (module: zlib/1.2.11-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/h/help2man/help2man-1.47.4-GCCcore-6.4.0.eb (module: help2man/1.47.4-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/b/Bison/Bison-3.0.4-GCCcore-6.4.0.eb (module: Bison/3.0.4-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/f/flex/flex-2.6.4-GCCcore-6.4.0.eb (module: flex/2.6.4-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/b/binutils/binutils-2.28-GCCcore-6.4.0.eb (module: binutils/2.28-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-GCCcore-6.4.0.eb (module: Autoconf/2.69-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/a/Automake/Automake-1.15.1-GCCcore-6.4.0.eb (module: Automake/1.15.1-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/g/GCC/GCC-6.4.0-2.28.eb (module: GCC/6.4.0-2.28)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/o/OpenBLAS/OpenBLAS-0.2.20-GCC-6.4.0-2.28.eb (module: OpenBLAS/0.2.20-GCC-6.4.0-2.28)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/l/libtool/libtool-2.4.6-GCCcore-6.4.0.eb (module: libtool/2.4.6-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/a/Autotools/Autotools-20170619-GCCcore-6.4.0.eb (module: Autotools/20170619-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/n/numactl/numactl-2.0.11-GCCcore-6.4.0.eb (module: numactl/2.0.11-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/h/hwloc/hwloc-1.11.8-GCCcore-6.4.0.eb (module: hwloc/1.11.8-GCCcore-6.4.0)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/o/OpenMPI/OpenMPI-2.1.2-GCC-6.4.0-2.28.eb (module: OpenMPI/2.1.2-GCC-6.4.0-2.28)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/g/gompi/gompi-2018a.eb (module: gompi/2018a)
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/f/FFTW/FFTW-3.3.7-gompi-2018a.eb (module: FFTW/3.3.7-gompi-(module: ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20)sr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018a-OpenBLAS-0.2.20.eb
 * [x] /usr/share/easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_easyconfigs-3.5.3-py2.7.egg/easybuild/easyconfigs/f/foss/foss-2018a.eb (module: foss/2018a)
 * [x] /tmp/eb-CEP096/tweaked_dep_easyconfigs/tbb-2017_U6-foss-2018a.eb (module: tbb/2017_U6-foss-2018a)
 * [ ] /tmp/eb-CEP096/tweaked_easyconfigs/Bowtie2-2.3.2-foss-2018a.eb (module: Bowtie2/2.3.2-foss-2018a)

And part of the build log (from eb Bowtie2-2.3.2-intel-2017a.eb --robot --try-toolchain=foss,2018a):

== 2018-05-03 12:48:46,142 environment.py:97 INFO Environment variable EBVARSCALAPACK_LIB_DIR set to /usr/share/easybuild/software/ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20/lib (previously undefined)
== 2018-05-03 12:48:46,142 environment.py:97 INFO Environment variable SCALAPACK_MT_STATIC_LIBS set to libscalapack.a,libopenblas.a,libgfortran.a,libpthread.a (previously undefined)
== 2018-05-03 12:48:46,142 environment.py:97 INFO Environment variable EBVARSCALAPACK_MT_STATIC_LIBS set to libscalapack.a,libopenblas.a,libgfortran.a,libpthread.a (previously undefined)
== 2018-05-03 12:48:46,142 environment.py:97 INFO Environment variable SCALAPACK_STATIC_LIBS set to libscalapack.a,libopenblas.a,libgfortran.a (previously undefined)
== 2018-05-03 12:48:46,142 environment.py:97 INFO Environment variable EBVARSCALAPACK_STATIC_LIBS set to libscalapack.a,libopenblas.a,libgfortran.a (previously undefined)
== 2018-05-03 12:48:46,143 easyblock.py:1430 INFO Using /tmp/eb-build/Bowtie2/2.3.2/foss-2018a/bowtie2-2.3.2/ as start dir
== 2018-05-03 12:48:46,143 build_log.py:233 INFO configuring...
== 2018-05-03 12:48:46,143 easyblock.py:2449 INFO Starting configure step
== 2018-05-03 12:48:46,144 easyblock.py:2455 INFO Running method configure_step part of step configure
== 2018-05-03 12:48:46,144 build_log.py:233 INFO building...
== 2018-05-03 12:48:46,145 easyblock.py:2449 INFO Starting build step
== 2018-05-03 12:48:46,145 easyblock.py:2455 INFO Running method build_step part of step build
== 2018-05-03 12:48:46,145 run.py:183 INFO running cmd:  make -j 2 EXTRA_FLAGS='-wd809' CC="gcc"  CPP="g++" CXX="g++"  RELEASE_FLAGS="-O2 -march=native -fPIC -std=c++11"
== 2018-05-03 12:48:46,251 build_log.py:158 ERROR EasyBuild crashed with an error (at easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/vsc_base-2.5.8-py2.7.egg/vsc/utils/exceptions.py:124 in __init__): cmd " make -j 2 EXTRA_FLAGS=
g++ -O2 -march=native -fPIC -std=c++11 -DCOMPILER_OPTIONS="\"-O2 -march=native -fPIC -std=c++11 -wd809 -DPOPCNT_CAPABILITY -DWITH_TBB -DNO_SPINLOCK -DWITH_QUEUELOCK=1\"" -wd809 -DPOPCNT_CAPABILITY -DWITH_TBB -DNO_SPINLOCK -DWITH_QUEUELOCK=1 \
        -fno-strict-aliasing -DBOWTIE2_VERSION="\"`cat VERSION`\"" -DBUILD_HOST="\"`hostname`\"" -DBUILD_TIME="\"`date`\"" -DCOMPILER_VERSION="\"`g++ -v 2>&1 | tail -1`\"" -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE  -DBOWTIE_MM  -
         -I third_party \
        -o bowtie2-build-s bt2_build.cpp \
        ccnt_lut.cpp ref_read.cpp alphabet.cpp shmem.cpp edit.cpp bt2_idx.cpp bt2_io.cpp bt2_util.cpp reference.cpp ds.cpp multikey_qsort.cpp limit.cpp random_source.cpp diff_sample.cpp bowtie_build_main.cpp \
        -lz -lpthread -ltbb -ltbbmalloc_proxy
g++ -O2 -march=native -fPIC -std=c++11 -DCOMPILER_OPTIONS="\"-O2 -march=native -fPIC -std=c++11 -wd809 -DPOPCNT_CAPABILITY -DWITH_TBB -DNO_SPINLOCK -DWITH_QUEUELOCK=1\"" -wd809 -DPOPCNT_CAPABILITY -DWITH_TBB -DNO_SPINLOCK -DWITH_QUEUELOCK=1 \
        -fno-strict-aliasing -DBOWTIE2_VERSION="\"`cat VERSION`\"" -DBUILD_HOST="\"`hostname`\"" -DBUILD_TIME="\"`date`\"" -DCOMPILER_VERSION="\"`g++ -v 2>&1 | tail -1`\"" -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE  -DBOWTIE_MM  -
         -I third_party \
        -o bowtie2-build-l bt2_build.cpp \
        ccnt_lut.cpp ref_read.cpp alphabet.cpp shmem.cpp edit.cpp bt2_idx.cpp bt2_io.cpp bt2_util.cpp reference.cpp ds.cpp multikey_qsort.cpp limit.cpp random_source.cpp diff_sample.cpp bowtie_build_main.cpp \
        -lz -lpthread -ltbb -ltbbmalloc_proxy
g++: error: unrecognized command line option ‘-wd809’
make: *** [bowtie2-build-s] Error 1
make: *** Waiting for unfinished jobs....
g++: error: unrecognized command line option ‘-wd809’
make: *** [bowtie2-build-l] Error 1
 (at easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_framework-3.5.3-py2.7.egg/easybuild/tools/run.py:481 in parse_cmd_output)

I do get a different cannot find -lz error when trying the foss-2017a EasyConfig with --try-toolchain=foss,2018a: (command eb Bowtie2-2.3.2-foss-2017a.eb --robot --try-toolchain=foss,2018a):

      diff_sample.h:181:6: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
      if(!diffs[d2]) diffCnt++; diffs[d2] = true;
      ^~
diff_sample.h:181:32: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’
      if(!diffs[d2]) diffCnt++; diffs[d2] = true;
                                ^~~~~
/usr/share/easybuild/software/binutils/2.28-GCCcore-6.4.0/bin/ld.gold: error: cannot find -lz
collect2: error: ld returned 1 exit status
make: *** [bowtie2-build-l] Error 1
make: *** Waiting for unfinished jobs....
/usr/share/easybuild/software/binutils/2.28-GCCcore-6.4.0/bin/ld.gold: error: cannot find -lz
collect2: error: ld returned 1 exit status
make: *** [bowtie2-build-s] Error 1
 (at easybuild/software/EasyBuild/3.5.3/lib/python2.7/site-packages/easybuild_framework-3.5.3-py2.7.egg/easybuild/tools/run.py:481 in parse_cmd_output)

In the meantime I'm going to try building the foss 2017a toolchain to see what happens.

Thank you!