search

easybuilders / easybuild-framework

EasyBuild is a software installation framework in Python that allows you to install software in a structured and robust way.
https://easybuild.io
GNU General Public License v2.0
152 stars 202 forks source link

Building COSMO Dycore fails in configure due to wrong directory generation with EB 4.2.0 / EB 4.2.1 #3361

Open omlins opened 4 years ago

omlins commented 4 years ago

It fails with this ERROR:

CMake Error: The source directory "/apps/daint/UES/omlins/cosmo_pe1908_keepsrc/Dycore/551f6b4/dycore" does not exist.

The correct folder would be:

/apps/daint/UES/omlins/cosmo_pe1908_keepsrc/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/dycore/

i.e. CrayGNU-19.08-cuda-10.1-crclim-double is missing.

It works with EB <4.2.0. The shell output of the complete failing use case is here:

$ ssh -X daint

omlins@daint103:~> export EASYBUILD_BUILDPATH=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
omlins@daint103:~> export EASYBUILD_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
omlins@daint103:~> export EB_CUSTOM_REPOSITORY=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild
omlins@daint103:~> module load daint-gpu
omlins@daint103:~> module unload xalt
omlins@daint103:~> module load cdt/19.08
Switching to atp/2.1.3.
Switching to cce/9.0.1.
Switching to cray-libsci/19.06.1.
Switching to cray-mpich/7.7.9.
Switching to craype/2.6.0.
Switching to modules/3.2.11.2.
Switching to perftools-base/7.1.0.
Switching to pmi/5.0.14.
omlins@daint103:~> module load EasyBuild-custom/cscs
omlins@daint103:~> module switch EasyBuild EasyBuild/4.2.1 
alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari
atp/2.1.3
cce/9.0.1
cdt/19.08
cray-libsci/19.06.1
cray-mpich/7.7.9
craype/2.6.0
craype-haswell
craype-network-aries
daint-gpu
dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari
dvs/2.12_2.2.151-7.0.1.1_5.6__g7eb5e703
EasyBuild/4.2.0
EasyBuild-custom/cscs
gni-headers/5.0.12.0-7.0.1.1_6.7__g3b1768f.ari
job/2.2.4-7.0.1.1_3.8__g36b56f4.ari
modules/3.2.11.2
perftools-base/7.1.0
pmi/5.0.14
PrgEnv-cray/6.0.5
rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari
slurm/20.02.2-1
udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari
ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari
xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari
omlins@daint103:~> cd /apps/daint/UES/omlins/cosmo_pe1908_keepsrc/
Boost/           CrayCCE/         grib_api/        production/
bzip2/           CrayGNU/         JasPer/          Serialbox/
CMake/           Dycore/          libgrib1_crclim/ software/
COSMO_pompa/     ebfiles_repo/    modules/         STELLA/
omlins@daint103:~> mkdir /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
omlins@daint103:~> cd /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1> git clone https://github.com/eth-cscs/production.git
Cloning into 'production'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 42865 (delta 13), reused 11 (delta 4), pack-reused 42837
Receiving objects: 100% (42865/42865), 7.74 MiB | 10.71 MiB/s, done.
Resolving deltas: 100% (31923/31923), done.
Checking out files: 100% (4601/4601), done.
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1> cd production/
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production> eb Dycore-551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.eb -r --disable-cleanup-tmpdir --disable-cleanup-builddir && eb COSMO_pompa-f9916ba-28.11.19-CrayCCE-19.08-cuda-10.0-crclim-double.eb -r --disable-cleanup-tmpdir --disable-cleanup-builddir
== temporary log file in case of crash /run/user/24345/easybuild/tmp/eb-Yx5j68/easybuild-I7OUpB.log

WARNING: Deprecated functionality, will no longer work in v5.0: Use of 'dummy' toolchain is deprecated, use 'system' toolchain instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information

== resolving dependencies ...

WARNING: Deprecated functionality, will no longer work in v5.0: Use of 'dummy' toolchain is deprecated, use 'system' toolchain instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information

== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/c/CMake/CMake-3.12.0.eb
== building and installing CMake/.3.12.0...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully (took 5 min 2 sec)
== Results of the build can be found in the log file(s) /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/easybuild/easybuild-CMake-3.12.0-20200603.172053.log
== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/c/CrayGNU/CrayGNU-19.08.eb
== building and installing CrayGNU/.19.08...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully (took 3 sec)
== Results of the build can be found in the log file(s) /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CrayGNU/19.08/easybuild/easybuild-CrayGNU-19.08-20200603.172057.log
== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/b/bzip2/bzip2-1.0.6-CrayGNU-19.08.eb
== building and installing bzip2/.1.0.6-CrayGNU-19.08...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully (took 10 sec)
== Results of the build can be found in the log file(s) /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/easybuild/easybuild-bzip2-1.0.6-20200603.172107.log
== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/b/Boost/Boost-1.67.0-CrayGNU-19.08.eb
== building and installing Boost/1.67.0-CrayGNU-19.08...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully (took 4 min 0 sec)
== Results of the build can be found in the log file(s) /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/easybuild/easybuild-Boost-1.67.0-20200603.172507.log
== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/s/Serialbox/Serialbox-2.4.1-CrayGNU-19.08.eb
== building and installing Serialbox/2.4.1-CrayGNU-19.08...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully (took 51 sec)
== Results of the build can be found in the log file(s) /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/easybuild/easybuild-Serialbox-2.4.1-20200603.172559.log
== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/s/STELLA/STELLA-4a5f9c5-CrayGNU-19.08-crclim-double.eb
== building and installing STELLA/4a5f9c5-CrayGNU-19.08-crclim-double...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully (took 1 min 52 sec)
== Results of the build can be found in the log file(s) /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/easybuild/easybuild-STELLA-4a5f9c5-20200603.172752.log
== processing EasyBuild easyconfig /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/d/Dycore/Dycore-551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.eb
== building and installing Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== FAILED: Installation ended unsuccessfully (build directory: /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double): build failed (first 300 chars): cmd " cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_C (took 3 sec)
== Results of the build can be found in the log file(s) /run/user/24345/easybuild/tmp/eb-Yx5j68/easybuild-Dycore-551f6b4-20200603.172752.aPLVW.log
ERROR: Build of /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/d/Dycore/Dycore-551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.eb failed (err: 'build failed (first 300 chars): cmd " cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER=\'cc\' -DCMAKE_Fortran_FLAGS=\'-O2 -fno-math-errno\' -DCMAKE_CXX_FLAGS=\'-O2 -fno-math-errno\' -DCMAKE_CXX_COMPILER=\'CC\' -DCMAKE_Fortran_C')
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production> module list
Currently Loaded Modulefiles:
  1) modules/3.2.11.2(default)
  2) cce/9.0.1(default)
  3) craype-haswell
  4) craype-network-aries
  5) craype/2.6.0(default)
  6) cray-libsci/19.06.1(default)
  7) udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari
  8) ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari
  9) pmi/5.0.14(default)
 10) dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari
 11) gni-headers/5.0.12.0-7.0.1.1_6.7__g3b1768f.ari
 12) xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari
 13) job/2.2.4-7.0.1.1_3.8__g36b56f4.ari
 14) dvs/2.12_2.2.151-7.0.1.1_5.6__g7eb5e703
 15) alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari
 16) rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari
 17) atp/2.1.3(default)
 18) perftools-base/7.1.0(default)
 19) PrgEnv-cray/6.0.5
 20) cray-mpich/7.7.9(default)
 21) slurm/20.02.2-1
 22) daint-gpu
 23) cdt/19.08
 24) EasyBuild/4.2.1
 25) EasyBuild-custom/cscs
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production> eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
allow-loaded-modules      (F) = ddt, EasyBuild-custom, EasyBuild, xalt
buildpath                 (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
containerpath             (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/containers
external-modules-metadata (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/cray_external_modules_metadata-19.10.cfg
hide-deps                 (F) = absl, ANTLR, APR, APR-util, Autoconf, Automake, Autotools, Bison, Bazel, cloudpickle, CMake, configurable-http-proxy, Coreutils, Cube, CUDA, Doxygen, Eigen, FLTK, GCCcore, GL2PS, glmnet, GLPK, GLib, GPGME, GTS, GraphicsMagick, help2man, IPython, JasPer, LLVM, LOKI, LVM2, Loki, Libassuan, LibTIFF, Libpgp-error, M4, Mako, Mesa, NASM, NASM, OPARI2, OTF2, OpenPGM, PCRE, PDT, PIL, Pillow, PyGTS, PyQt, PyYAML, PyZMQ, Python-Xlib, Python-bare, Qhull, Qt, SCOTCH, SCons, SIONlib, SIP, SIRIUS, SQLite, SWIG, Serf, Szip, Tcl, Tk, UDUNITS, X11, XZ, ZeroMQ, arpack-ng, backports.weakref, binutils, bokeh, byacc, bzip2, cairo, cURL, cuDNN, expat, flex, fontconfig, freetype, funcsigs, gc, gettext, gnuplot, go, gperf, guile, hwloc, inputproto, jemalloc, kbproto, libGLU, libgpuarray, libQGLViewer, libspatialindex, libX11, libXau, libXdmcp, libXext, libXrender, libcerf, libdrm, libevent, libffi, libgd, libiberty, libjpeg-turbo, libjpeg-turbo, libpng, libpthread-stubs, libreadline, libsodium, libtool, libunistring, libunwind, libutempter, libxcb, libxml2, libxshmfence, libyaml, make, minieigen, mock, mxml, ncurses, nettle, networkx, nodejs, nose-parameterized, numactl, parameterized, pixman, pkg-config, ploticus, popt, prereq, protobuf, protobuf-core, qrupdate, renderproto, runc, scikit-image, scikit-learn, util-linux, Werkzeug, wheel, xcb-proto, xextproto, xorg-macros, xproto, xtrans, zlib
hide-toolchains           (F) = CrayCCE, CrayGNU, CrayIntel, CrayPGI, GCCcore, gmvapich2, gmvolf, foss, fosscuda, gompi
include-easyblocks        (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyblocks/*.py
installpath               (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
module-syntax             (F) = Tcl
modules-tool              (E) = EnvironmentModulesC
optarch                   (E) = haswell
packagepath               (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/packages
prefix                    (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1
repositorypath            (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/ebfiles_repo
robot-paths               (E) = /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs/, /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production/easybuild/easyconfigs, /apps/common/UES/easybuild/software/EasyBuild/4.2.1/easybuild/easyconfigs
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production>

Here is the log file:

omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production> cat /run/user/24345/easybuild/tmp/eb-Yx5j68/easybuild-Dycore-551f6b4-20200603.172752.aPLVW.log
== 2020-06-03 17:27:52,341 easyblock.py:280 INFO This is EasyBuild 4.2.1 (framework: 4.2.1, easyblocks: 4.2.1) on host daint103.
== 2020-06-03 17:27:52,341 easyblock.py:286 INFO This is easyblock CMakeMake from module easybuild.easyblocks.generic.cmakemake (/apps/common/UES/easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/easyblocks/generic/cmakemake.py)
== 2020-06-03 17:27:52,341 easyblock.py:843 INFO Build dir set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double
== 2020-06-03 17:27:52,341 easyblock.py:903 INFO Software install dir set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double
== 2020-06-03 17:27:52,341 easyblock.py:908 INFO Module install dir set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/all
== 2020-06-03 17:27:52,341 easyblock.py:265 INFO Init completed for application name Dycore version 551f6b4
== 2020-06-03 17:27:52,341 easyblock.py:3208 INFO Obtained application instance of for Dycore (easyblock: CMakeMake)
== 2020-06-03 17:27:52,342 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:52,342 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:52,342 one.py:159 INFO Skipping reformatting value for parameter 'toolchain'
== 2020-06-03 17:27:52,342 one.py:159 INFO Skipping reformatting value for parameter 'toolchainopts'
== 2020-06-03 17:27:52,343 filetools.py:1495 INFO Creating directory /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476 (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,343 easyblock.py:3385 INFO Dumped easyconfig instance to /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476/Dycore-551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.eb
== 2020-06-03 17:27:52,343 filetools.py:1495 INFO Creating directory /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476/easyblocks (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,344 filetools.py:2014 INFO /apps/common/UES/easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/easyblocks/generic/cmakemake.py copied to /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476/easyblocks/cmakemake.py
== 2020-06-03 17:27:52,344 easyblock.py:3400 INFO Dumped easyblock cmakemake.py required for reproduction to /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476/easyblocks
== 2020-06-03 17:27:52,344 filetools.py:2014 INFO /apps/common/UES/easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/easyblocks/generic/configuremake.py copied to /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476/easyblocks/configuremake.py
== 2020-06-03 17:27:52,344 easyblock.py:3400 INFO Dumped easyblock configuremake.py required for reproduction to /run/user/24345/easybuild/tmp/eb-Yx5j68/reprod_20200603172752_25476/easyblocks
== 2020-06-03 17:27:52,344 easyblock.py:1635 INFO Number of iterations to perform for central part of installation procedure: 1
== 2020-06-03 17:27:52,345 build_log.py:265 INFO building and installing Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double...
== 2020-06-03 17:27:52,345 filetools.py:1597 INFO Lock /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock does not exist
== 2020-06-03 17:27:52,345 filetools.py:1538 INFO Creating lock at /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock...
== 2020-06-03 17:27:52,345 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,346 filetools.py:1547 INFO Lock created: /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock
== 2020-06-03 17:27:52,346 build_log.py:265 INFO fetching files...
== 2020-06-03 17:27:52,346 easyblock.py:2968 INFO Starting fetch step
== 2020-06-03 17:27:52,346 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:52,346 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:52,347 easyblock.py:2974 INFO Running method fetch_step part of step fetch
== 2020-06-03 17:27:52,347 easyblock.py:1744 WARNING Easyconfig does not specify an EasyBuild-version (key 'easybuild_version')! Assuming the latest version
== 2020-06-03 17:27:52,347 tools.py:266 INFO Found location to 'eb' script via $EB_SCRIPT_PATH: /apps/common/UES/easybuild/software/EasyBuild/4.2.1/bin/eb
== 2020-06-03 17:27:52,347 tools.py:280 INFO Also considering installation prefix /apps/common/UES/easybuild/software/EasyBuild/4.2.1 (determined via path to 'eb' script)...
== 2020-06-03 17:27:52,351 easyblock.py:671 INFO Found file /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz at /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-03 17:27:52,351 easyblock.py:671 INFO Found file /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz at /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-03 17:27:52,351 easyblock.py:671 INFO Found file /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz at /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-03 17:27:52,351 easyblock.py:394 INFO Added sources: [{'path': '/apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz', 'cmd': None, 'finalpath': '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double', 'name': '/apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz', 'checksum': None}]
== 2020-06-03 17:27:52,351 easyblock.py:1785 INFO no patches provided
== 2020-06-03 17:27:52,424 easyblock.py:1795 INFO md5 checksum for /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz: 9e6ee432258501c4dd8f25e8bacc051c
== 2020-06-03 17:27:52,428 easyblock.py:1795 INFO sha256 checksum for /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz: a81d4c2223c97e5d6f924c34e6852e1e386ce7a38de635f3a7bc05515ea94a9f
== 2020-06-03 17:27:52,429 easyblock.py:1830 INFO Checking dirs that need to be created: ['/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double', '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/all/Dycore', '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/devel/Dycore']
== 2020-06-03 17:27:52,429 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,430 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/all/Dycore (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,430 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/devel/Dycore (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,431 configuremake.py:93 INFO config.guess version: 2018-08-29 (last updated: 2020-05-25T16:05:49.317540, SHA256 checksum: c02eb9cc55c86cfd1e9a794e548d25db5c9539e7b2154beb649bc6e2cbffc74c)
== 2020-06-03 17:27:52,431 configuremake.py:148 INFO Found config.guess at /apps/common/easybuild/sources/generic/eb_v4.2.1/ConfigureMake/config.guess
== 2020-06-03 17:27:52,431 build_log.py:265 INFO creating build dir, resetting environment...
== 2020-06-03 17:27:52,431 easyblock.py:2968 INFO Starting ready step
== 2020-06-03 17:27:52,431 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:52,431 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:52,431 easyblock.py:2974 INFO Running method check_readiness_step part of step ready
== 2020-06-03 17:27:52,441 easyblock.py:1654 INFO Setting parallelism: 20
== 2020-06-03 17:27:52,456 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:52,456 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:52,456 easyblock.py:1693 WARNING Loaded modules detected: ['modules/3.2.11.2', 'cce/9.0.1', 'craype-haswell', 'craype-network-aries', 'craype/2.6.0', 'cray-libsci/19.06.1', 'udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari', 'ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari', 'pmi/5.0.14', 'dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari', 'gni-headers/5.0.12.0-7.0.1.1_6.7__g3b1768f.ari', 'xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari', 'job/2.2.4-7.0.1.1_3.8__g36b56f4.ari', 'dvs/2.12_2.2.151-7.0.1.1_5.6__g7eb5e703', 'alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari', 'rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari', 'atp/2.1.3', 'perftools-base/7.1.0', 'PrgEnv-cray/6.0.5', 'cray-mpich/7.7.9', 'slurm/20.02.2-1', 'daint-gpu', 'cdt/19.08', 'EasyBuild/4.2.1', 'EasyBuild-custom/cscs']
== 2020-06-03 17:27:52,456 easyblock.py:2974 INFO Running method make_builddir part of step ready
== 2020-06-03 17:27:52,457 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:52,457 easyblock.py:2974 INFO Running method reset_env part of step ready
== 2020-06-03 17:27:52,458 easyblock.py:2974 INFO Running method handle_iterate_opts part of step ready
== 2020-06-03 17:27:52,459 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:52,459 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:52,459 build_log.py:265 INFO unpacking...
== 2020-06-03 17:27:52,459 easyblock.py:2968 INFO Starting source step
== 2020-06-03 17:27:52,459 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:52,459 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:52,459 easyblock.py:2974 INFO Running method checksum_step part of step source
== 2020-06-03 17:27:52,459 easyblock.py:1846 INFO Checksum verification for /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz using None passed.
== 2020-06-03 17:27:52,460 easyblock.py:2974 INFO Running method extract_step part of step source
== 2020-06-03 17:27:52,460 easyblock.py:1933 INFO Unpacking source /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-03 17:27:52,460 run.py:222 INFO running cmd: tar xzf /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz 
== 2020-06-03 17:27:53,075 build_log.py:265 INFO patching...
== 2020-06-03 17:27:53,075 easyblock.py:2968 INFO Starting patch step
== 2020-06-03 17:27:53,076 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:53,076 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:53,076 easyblock.py:2974 INFO Running method patch_step part of step patch
== 2020-06-03 17:27:53,076 build_log.py:265 INFO preparing...
== 2020-06-03 17:27:53,076 easyblock.py:2968 INFO Starting prepare step
== 2020-06-03 17:27:53,076 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:53,076 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:53,077 easyblock.py:2974 INFO Running method prepare_step part of step prepare
== 2020-06-03 17:27:53,275 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:53,275 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:53,321 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:53,322 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,353 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/opt/gcc/8.3.0/snos/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,353 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,366 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,367 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,388 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,389 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,414 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,415 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,449 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,450 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,551 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,552 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,552 environment.py:97 INFO Environment variable EBROOTCUDA set to /opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7 (previously undefined)
== 2020-06-03 17:27:54,552 environment.py:97 INFO Environment variable EBVERSIONCUDA set to 10.1.105_3.27-7.0.1.1_4.1__ga311ce7 (previously undefined)
== 2020-06-03 17:27:54,569 toolchain.py:703 INFO List of toolchain dependency modules and toolchain definition match!
== 2020-06-03 17:27:54,599 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,599 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,615 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-03 17:27:54,615 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-03 17:27:54,619 variables.py:201 WARNING flags_for_subdirs: directory /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib64 was not found
== 2020-06-03 17:27:54,622 variables.py:201 WARNING flags_for_subdirs: directory /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib64 was not found
== 2020-06-03 17:27:54,622 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/include was not found
== 2020-06-03 17:27:54,622 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/lib64 was not found
== 2020-06-03 17:27:54,622 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/lib was not found
== 2020-06-03 17:27:54,622 variables.py:201 WARNING flags_for_subdirs: directory /opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib was not found
== 2020-06-03 17:27:54,622 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib64 was not found
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable BLAS_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable EBVARBLAS_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable BLAS_LAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable BLAS_LAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable BLAS_LAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,624 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable BLAS_LAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable BLAS_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable EBVARBLAS_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable BLAS_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable EBVARBLAS_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable BLAS_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable EBVARBLAS_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable CC set to cc (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable EBVARCC set to cc (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable CFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,625 environment.py:97 INFO Environment variable EBVARCFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable CPPFLAGS set to -I/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/include -I/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/include -I/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable EBVARCPPFLAGS set to -I/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/include -I/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/include -I/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable CXX set to CC (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable EBVARCXX set to CC (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable CXXFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable EBVARCXXFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable F77 set to ftn (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable EBVARF77 set to ftn (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable F90 set to ftn (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable EBVARF90 set to ftn (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable F90FLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,626 environment.py:97 INFO Environment variable EBVARF90FLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable FC set to ftn (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable EBVARFC set to ftn (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable FCFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable EBVARFCFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable FFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable EBVARFFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable LAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable EBVARLAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable LAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable EBVARLAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable LAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable EBVARLAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,627 environment.py:97 INFO Environment variable LAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable LDFLAGS set to -L/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib -L/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64 -L/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLDFLAGS set to -L/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib -L/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64 -L/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable LIBBLAS set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLIBBLAS set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable LIBBLAS_MT set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLIBBLAS_MT set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable LIBLAPACK set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLIBLAPACK set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable LIBLAPACK_MT set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLIBLAPACK_MT set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable LIBLAPACK_MT_ONLY set to  (previously undefined)
== 2020-06-03 17:27:54,628 environment.py:97 INFO Environment variable EBVARLIBLAPACK_MT_ONLY set to  (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable LIBLAPACK_ONLY set to  (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable EBVARLIBLAPACK_ONLY set to  (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable MPICC set to cc (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable EBVARMPICC set to cc (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable MPICXX set to CC (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable EBVARMPICXX set to CC (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable MPIF77 set to ftn (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable EBVARMPIF77 set to ftn (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable MPIF90 set to ftn (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable EBVARMPIF90 set to ftn (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable MPIFC set to ftn (previously undefined)
== 2020-06-03 17:27:54,629 environment.py:97 INFO Environment variable EBVARMPIFC set to ftn (previously undefined)
== 2020-06-03 17:27:54,630 environment.py:97 INFO Environment variable OPTFLAGS set to -O2 (previously undefined)
== 2020-06-03 17:27:54,630 environment.py:97 INFO Environment variable EBVAROPTFLAGS set to -O2 (previously undefined)
== 2020-06-03 17:27:54,630 environment.py:97 INFO Environment variable PRECFLAGS set to -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,630 environment.py:97 INFO Environment variable EBVARPRECFLAGS set to -fno-math-errno (previously undefined)
== 2020-06-03 17:27:54,630 environment.py:97 INFO Environment variable CRAYPE_LINK_TYPE set to dynamic (previous value: 'dynamic')
== 2020-06-03 17:27:54,630 easyblock.py:1554 INFO Using /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/ as start dir
== 2020-06-03 17:27:54,630 build_log.py:265 INFO configuring...
== 2020-06-03 17:27:54,630 easyblock.py:2968 INFO Starting configure step
== 2020-06-03 17:27:54,631 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-03 17:27:54,631 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-03 17:27:54,631 easyblock.py:2974 INFO Running method configure_step part of step configure
== 2020-06-03 17:27:54,631 environment.py:97 INFO Environment variable CMAKE_INCLUDE_PATH set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/include:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/include (previously undefined)
== 2020-06-03 17:27:54,631 environment.py:97 INFO Environment variable CMAKE_LIBRARY_PATH set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previously undefined)
== 2020-06-03 17:27:54,631 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/easybuild_obj (parents: True, set_gid: False, sticky: False)
== 2020-06-03 17:27:54,632 run.py:222 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_COMPILER='ftn' -DCMAKE_C_FLAGS='-O2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DBOOST_ROOT=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08 -DBoost_NO_SYSTEM_PATHS=ON -DBoost_NO_BOOST_CMAKE=ON -DCMAKE_BUILD_TYPE=Release -DSINGLEPRECISION=OFF -DCUDA_BACKEND=ON -DSTELLA_DIR=${EBROOTSTELLA} -DBITREPRO=OFF  /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/../dycore 
== 2020-06-03 17:27:55,450 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/base/exceptions.py:124 in __init__): cmd " cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_COMPILER='ftn' -DCMAKE_C_FLAGS='-O2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DBOOST_ROOT=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08 -DBoost_NO_SYSTEM_PATHS=ON -DBoost_NO_BOOST_CMAKE=ON -DCMAKE_BUILD_TYPE=Release -DSINGLEPRECISION=OFF -DCUDA_BACKEND=ON -DSTELLA_DIR=${EBROOTSTELLA} -DBITREPRO=OFF  /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/../dycore" exited with exit code 1 and output:
CMake Error: The source directory "/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/dycore" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
 (at easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/tools/run.py:533 in parse_cmd_output)
== 2020-06-03 17:27:55,450 filetools.py:1605 INFO Removing lock /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock...
== 2020-06-03 17:27:55,451 filetools.py:330 INFO Path /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock successfully removed.
== 2020-06-03 17:27:55,451 filetools.py:1609 INFO Lock removed: /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock
== 2020-06-03 17:27:55,451 easyblock.py:3238 WARNING build failed (first 300 chars): cmd " cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_C
== 2020-06-03 17:27:55,451 easyblock.py:295 INFO Closing log for application name Dycore version 551f6b4
omlins@daint103:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/production>
boegel commented 4 years ago

@omlins Can you share the easyconfig file that is used for the failing installation?

This likely has something to do with the switch we made in the CMakeMake easyblock to do out-of-source builds by default, see https://github.com/easybuilders/easybuild-easyblocks/pull/1933 .

If that's indeed the culprit, you should be able to fix the problem using separate_build_dir = False in the easyconfig file.

Do let us know if this helps.

omlins commented 4 years ago

Thanks @boegel. Setting separate_build_dir = False did however not make it succeed building.

Here is the easyconfig file of the failing installation (before there was separate_build_dir = True):

omlins@daint106:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc/production> cat easybuild/easyconfigs/d/Dycore/Dycore-551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.eb
# omlins, kraushm, jg, havogt (cscs)
easyblock = 'CMakeMake'

name = 'Dycore'
version = '551f6b4'
versionsuffix = '-cuda-10.1-crclim-double'

homepage = "https://github.com/C2SM-RCM/cosmo-pompa/tree/master/dycore (-b crclim)"
description = 'cosmo-pompa-dycore'

toolchain = {'name': 'CrayGNU', 'version': '19.08'}
toolchainopts = {'verbose': False}

# repo is private (+ no official release)
sources = [{'filename': '/apps/common/UES/easybuild/sources/d/%(name)s/%(name)s_%(version)s.tar.gz'}]

builddependencies = [
    ('CMake', '3.12.0', '', True),
    ('cudatoolkit/10.1.105_3.27-7.0.1.1_4.1__ga311ce7', EXTERNAL_MODULE),
]
dependencies = [
    ('craype-accel-nvidia60', EXTERNAL_MODULE),
    ('STELLA', '4a5f9c5', '-crclim-double'),
]

configopts = "-DCMAKE_BUILD_TYPE=Release -DSINGLEPRECISION=OFF -DCUDA_BACKEND=ON -DSTELLA_DIR=${EBROOTSTELLA} -DBITREPRO=OFF "

srcdir = '../%(namelower)s'
separate_build_dir = False

moduleclass = 'devel'
omlins commented 4 years ago

It still says:

CMake Error: The source directory "/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/dycore" does not exist.

Here is the log:

omlins@daint106:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc/production> cat /run/user/24345/easybuild/tmp/eb-Q6cEN5/easybuild-Dycore-551f6b4-20200604.091758.sbRPB.log
== 2020-06-04 09:17:58,402 easyblock.py:280 INFO This is EasyBuild 4.2.1 (framework: 4.2.1, easyblocks: 4.2.1) on host daint106.
== 2020-06-04 09:17:58,402 easyblock.py:286 INFO This is easyblock CMakeMake from module easybuild.easyblocks.generic.cmakemake (/apps/common/UES/easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/easyblocks/generic/cmakemake.py)
== 2020-06-04 09:17:58,402 easyblock.py:843 INFO Build dir set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double
== 2020-06-04 09:17:58,402 easyblock.py:903 INFO Software install dir set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double
== 2020-06-04 09:17:58,403 easyblock.py:908 INFO Module install dir set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/all
== 2020-06-04 09:17:58,403 easyblock.py:265 INFO Init completed for application name Dycore version 551f6b4
== 2020-06-04 09:17:58,403 easyblock.py:3208 INFO Obtained application instance of for Dycore (easyblock: CMakeMake)
== 2020-06-04 09:17:58,403 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,403 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,404 one.py:159 INFO Skipping reformatting value for parameter 'toolchain'
== 2020-06-04 09:17:58,404 one.py:159 INFO Skipping reformatting value for parameter 'toolchainopts'
== 2020-06-04 09:17:58,404 filetools.py:1495 INFO Creating directory /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854 (parents: True, set_gid: False, sticky: False)
== 2020-06-04 09:17:58,405 easyblock.py:3385 INFO Dumped easyconfig instance to /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854/Dycore-551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.eb
== 2020-06-04 09:17:58,405 filetools.py:1495 INFO Creating directory /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854/easyblocks (parents: True, set_gid: False, sticky: False)
== 2020-06-04 09:17:58,405 filetools.py:2014 INFO /apps/common/UES/easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/easyblocks/generic/cmakemake.py copied to /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854/easyblocks/cmakemake.py
== 2020-06-04 09:17:58,405 easyblock.py:3400 INFO Dumped easyblock cmakemake.py required for reproduction to /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854/easyblocks
== 2020-06-04 09:17:58,406 filetools.py:2014 INFO /apps/common/UES/easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/easyblocks/generic/configuremake.py copied to /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854/easyblocks/configuremake.py
== 2020-06-04 09:17:58,406 easyblock.py:3400 INFO Dumped easyblock configuremake.py required for reproduction to /run/user/24345/easybuild/tmp/eb-Q6cEN5/reprod_20200604091758_5854/easyblocks
== 2020-06-04 09:17:58,406 easyblock.py:1635 INFO Number of iterations to perform for central part of installation procedure: 1
== 2020-06-04 09:17:58,406 build_log.py:265 INFO building and installing Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double...
== 2020-06-04 09:17:58,406 filetools.py:1597 INFO Lock /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock does not exist
== 2020-06-04 09:17:58,406 filetools.py:1538 INFO Creating lock at /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock...
== 2020-06-04 09:17:58,407 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock (parents: True, set_gid: False, sticky: False)
== 2020-06-04 09:17:58,407 filetools.py:1547 INFO Lock created: /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock
== 2020-06-04 09:17:58,407 build_log.py:265 INFO fetching files...
== 2020-06-04 09:17:58,407 easyblock.py:2968 INFO Starting fetch step
== 2020-06-04 09:17:58,407 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,407 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,408 easyblock.py:2974 INFO Running method fetch_step part of step fetch
== 2020-06-04 09:17:58,408 easyblock.py:1744 WARNING Easyconfig does not specify an EasyBuild-version (key 'easybuild_version')! Assuming the latest version
== 2020-06-04 09:17:58,408 tools.py:266 INFO Found location to 'eb' script via $EB_SCRIPT_PATH: /apps/common/UES/easybuild/software/EasyBuild/4.2.1/bin/eb
== 2020-06-04 09:17:58,408 tools.py:280 INFO Also considering installation prefix /apps/common/UES/easybuild/software/EasyBuild/4.2.1 (determined via path to 'eb' script)...
== 2020-06-04 09:17:58,409 easyblock.py:671 INFO Found file /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz at /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-04 09:17:58,409 easyblock.py:671 INFO Found file /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz at /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-04 09:17:58,409 easyblock.py:671 INFO Found file /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz at /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-04 09:17:58,409 easyblock.py:394 INFO Added sources: [{'path': '/apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz', 'cmd': None, 'finalpath': '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double', 'name': '/apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz', 'checksum': None}]
== 2020-06-04 09:17:58,409 easyblock.py:1785 INFO no patches provided
== 2020-06-04 09:17:58,414 easyblock.py:1795 INFO md5 checksum for /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz: 9e6ee432258501c4dd8f25e8bacc051c
== 2020-06-04 09:17:58,419 easyblock.py:1795 INFO sha256 checksum for /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz: a81d4c2223c97e5d6f924c34e6852e1e386ce7a38de635f3a7bc05515ea94a9f
== 2020-06-04 09:17:58,419 easyblock.py:1830 INFO Checking dirs that need to be created: ['/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double', '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/all/Dycore', '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/modules/devel/Dycore']
== 2020-06-04 09:17:58,420 configuremake.py:93 INFO config.guess version: 2018-08-29 (last updated: 2020-05-25T16:05:49.317540, SHA256 checksum: c02eb9cc55c86cfd1e9a794e548d25db5c9539e7b2154beb649bc6e2cbffc74c)
== 2020-06-04 09:17:58,420 configuremake.py:148 INFO Found config.guess at /apps/common/easybuild/sources/generic/eb_v4.2.1/ConfigureMake/config.guess
== 2020-06-04 09:17:58,420 build_log.py:265 INFO creating build dir, resetting environment...
== 2020-06-04 09:17:58,421 easyblock.py:2968 INFO Starting ready step
== 2020-06-04 09:17:58,421 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,421 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,421 easyblock.py:2974 INFO Running method check_readiness_step part of step ready
== 2020-06-04 09:17:58,430 run.py:222 INFO running cmd: ulimit -u 
== 2020-06-04 09:17:58,546 easyblock.py:1654 INFO Setting parallelism: 24
== 2020-06-04 09:17:58,559 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:17:58,560 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:17:58,560 easyblock.py:1693 WARNING Loaded modules detected: ['modules/3.2.11.2', 'cce/9.0.1', 'craype-haswell', 'craype-network-aries', 'craype/2.6.0', 'cray-libsci/19.06.1', 'udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari', 'ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari', 'pmi/5.0.14', 'dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari', 'gni-headers/5.0.12.0-7.0.1.1_6.7__g3b1768f.ari', 'xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari', 'job/2.2.4-7.0.1.1_3.8__g36b56f4.ari', 'dvs/2.12_2.2.151-7.0.1.1_5.6__g7eb5e703', 'alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari', 'rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari', 'atp/2.1.3', 'perftools-base/7.1.0', 'PrgEnv-cray/6.0.5', 'cray-mpich/7.7.9', 'slurm/20.02.2-1', 'daint-gpu', 'cdt/19.08', 'EasyBuild/4.2.1', 'EasyBuild-custom/cscs']
== 2020-06-04 09:17:58,560 easyblock.py:2974 INFO Running method make_builddir part of step ready
== 2020-06-04 09:17:58,560 easyblock.py:928 INFO Found old directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double
== 2020-06-04 09:17:58,607 filetools.py:330 INFO Path /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double successfully removed.
== 2020-06-04 09:17:58,607 easyblock.py:936 INFO Removed old directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double
== 2020-06-04 09:17:58,607 filetools.py:1495 INFO Creating directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double (parents: True, set_gid: False, sticky: False)
== 2020-06-04 09:17:58,607 easyblock.py:2974 INFO Running method reset_env part of step ready
== 2020-06-04 09:17:58,609 easyblock.py:2974 INFO Running method handle_iterate_opts part of step ready
== 2020-06-04 09:17:58,609 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,609 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,609 build_log.py:265 INFO unpacking...
== 2020-06-04 09:17:58,610 easyblock.py:2968 INFO Starting source step
== 2020-06-04 09:17:58,610 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,610 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,610 easyblock.py:2974 INFO Running method checksum_step part of step source
== 2020-06-04 09:17:58,610 easyblock.py:1846 INFO Checksum verification for /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz using None passed.
== 2020-06-04 09:17:58,610 easyblock.py:2974 INFO Running method extract_step part of step source
== 2020-06-04 09:17:58,610 easyblock.py:1933 INFO Unpacking source /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz
== 2020-06-04 09:17:58,611 run.py:222 INFO running cmd: tar xzf /apps/common/UES/easybuild/sources/d/Dycore/Dycore_551f6b4.tar.gz 
== 2020-06-04 09:17:58,931 build_log.py:265 INFO patching...
== 2020-06-04 09:17:58,932 easyblock.py:2968 INFO Starting patch step
== 2020-06-04 09:17:58,932 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,932 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,932 easyblock.py:2974 INFO Running method patch_step part of step patch
== 2020-06-04 09:17:58,932 build_log.py:265 INFO preparing...
== 2020-06-04 09:17:58,933 easyblock.py:2968 INFO Starting prepare step
== 2020-06-04 09:17:58,933 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:17:58,933 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:17:58,933 easyblock.py:2974 INFO Running method prepare_step part of step prepare
== 2020-06-04 09:17:58,977 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:17:58,978 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,062 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/opt/gcc/8.3.0/snos/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,062 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,077 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,077 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,101 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,101 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,130 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,130 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,166 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,167 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,275 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,275 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,276 environment.py:97 INFO Environment variable EBROOTCUDA set to /opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7 (previously undefined)
== 2020-06-04 09:18:00,276 environment.py:97 INFO Environment variable EBVERSIONCUDA set to 10.1.105_3.27-7.0.1.1_4.1__ga311ce7 (previously undefined)
== 2020-06-04 09:18:00,295 toolchain.py:703 INFO List of toolchain dependency modules and toolchain definition match!
== 2020-06-04 09:18:00,325 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,326 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,343 environment.py:97 INFO Environment variable LD_LIBRARY_PATH set to /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previous value: '/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin')
== 2020-06-04 09:18:00,343 environment.py:97 INFO Environment variable LD_PRELOAD set to  (previous value: '')
== 2020-06-04 09:18:00,347 variables.py:201 WARNING flags_for_subdirs: directory /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib64 was not found
== 2020-06-04 09:18:00,349 variables.py:201 WARNING flags_for_subdirs: directory /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib64 was not found
== 2020-06-04 09:18:00,350 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/include was not found
== 2020-06-04 09:18:00,350 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/lib64 was not found
== 2020-06-04 09:18:00,350 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/CMake/3.12.0/lib was not found
== 2020-06-04 09:18:00,350 variables.py:201 WARNING flags_for_subdirs: directory /opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib was not found
== 2020-06-04 09:18:00,350 variables.py:201 WARNING flags_for_subdirs: directory /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib64 was not found
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable BLAS_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable EBVARBLAS_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable BLAS_LAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable BLAS_LAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable BLAS_LAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable BLAS_LAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,352 environment.py:97 INFO Environment variable EBVARBLAS_LAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable BLAS_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable EBVARBLAS_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable BLAS_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable EBVARBLAS_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable BLAS_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable EBVARBLAS_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable CC set to cc (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable EBVARCC set to cc (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable CFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable EBVARCFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable CPPFLAGS set to -I/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/include -I/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/include -I/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,353 environment.py:97 INFO Environment variable EBVARCPPFLAGS set to -I/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/include -I/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/include -I/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable CXX set to CC (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable EBVARCXX set to CC (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable CXXFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable EBVARCXXFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable F77 set to ftn (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable EBVARF77 set to ftn (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable F90 set to ftn (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable EBVARF90 set to ftn (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable F90FLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable EBVARF90FLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable FC set to ftn (previously undefined)
== 2020-06-04 09:18:00,354 environment.py:97 INFO Environment variable EBVARFC set to ftn (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable FCFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable EBVARFCFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable FFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable EBVARFFLAGS set to -O2 -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable LAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable EBVARLAPACK_INC_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable LAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable EBVARLAPACK_LIB_DIR set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable LAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable EBVARLAPACK_MT_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable LAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable EBVARLAPACK_STATIC_LIBS set to  (previously undefined)
== 2020-06-04 09:18:00,355 environment.py:97 INFO Environment variable LDFLAGS set to -L/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib -L/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64 -L/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLDFLAGS set to -L/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib -L/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64 -L/opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable LIBBLAS set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLIBBLAS set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable LIBBLAS_MT set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLIBBLAS_MT set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable LIBLAPACK set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLIBLAPACK set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable LIBLAPACK_MT set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLIBLAPACK_MT set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable LIBLAPACK_MT_ONLY set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLIBLAPACK_MT_ONLY set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable LIBLAPACK_ONLY set to  (previously undefined)
== 2020-06-04 09:18:00,356 environment.py:97 INFO Environment variable EBVARLIBLAPACK_ONLY set to  (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable MPICC set to cc (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable EBVARMPICC set to cc (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable MPICXX set to CC (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable EBVARMPICXX set to CC (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable MPIF77 set to ftn (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable EBVARMPIF77 set to ftn (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable MPIF90 set to ftn (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable EBVARMPIF90 set to ftn (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable MPIFC set to ftn (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable EBVARMPIFC set to ftn (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable OPTFLAGS set to -O2 (previously undefined)
== 2020-06-04 09:18:00,357 environment.py:97 INFO Environment variable EBVAROPTFLAGS set to -O2 (previously undefined)
== 2020-06-04 09:18:00,358 environment.py:97 INFO Environment variable PRECFLAGS set to -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,358 environment.py:97 INFO Environment variable EBVARPRECFLAGS set to -fno-math-errno (previously undefined)
== 2020-06-04 09:18:00,358 environment.py:97 INFO Environment variable CRAYPE_LINK_TYPE set to dynamic (previous value: 'dynamic')
== 2020-06-04 09:18:00,358 easyblock.py:1554 INFO Using /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/ as start dir
== 2020-06-04 09:18:00,358 build_log.py:265 INFO configuring...
== 2020-06-04 09:18:00,358 easyblock.py:2968 INFO Starting configure step
== 2020-06-04 09:18:00,359 mpi.py:119 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'MPICH'
== 2020-06-04 09:18:00,359 mpi.py:291 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'cmd': 'xxx_command_xxx', 'nr_ranks': 1})
== 2020-06-04 09:18:00,359 easyblock.py:2974 INFO Running method configure_step part of step configure
== 2020-06-04 09:18:00,359 environment.py:97 INFO Environment variable CMAKE_INCLUDE_PATH set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/include:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/include:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/include:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/include (previously undefined)
== 2020-06-04 09:18:00,359 environment.py:97 INFO Environment variable CMAKE_LIBRARY_PATH set to /opt/cray/pe/libsci/19.06.1/GNU/8.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/lib64:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/STELLA/4a5f9c5-CrayGNU-19.08-crclim-double/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08/lib:/opt/cray/pe/libsci_acc/19.10.1/GNU/7.1/x86_64/lib:/opt/nvidia/cudatoolkit10/10.1.105_3.27-7.0.1.1_4.1__ga311ce7/extras/CUPTI/lib64:/opt/cray/pe/mpt/7.7.9/gni/mpich-gnu/8.2/lib:/opt/cray/pe/perftools/7.1.0/lib64:/opt/cray/rca/2.2.20-7.0.1.1_4.9__g8e3fb5b.ari/lib64:/opt/cray/alps/6.6.56-7.0.1.1_4.10__g2e60a7e4.ari/lib64:/opt/cray/xpmem/2.2.19-7.0.1.1_3.7__gdcf436c.ari/lib64:/opt/cray/dmapp/7.1.1-7.0.1.1_4.8__g38cf134.ari/lib64:/opt/cray/pe/pmi/5.0.14/lib64:/opt/cray/ugni/6.0.14.0-7.0.1.1_7.10__ge78e5b0.ari/lib64:/opt/cray/udreg/2.3.2-7.0.1.1_3.9__g8175d3d.ari/lib64:/apps/daint/UES/jenkins/7.0.UP01/gpu/easybuild/software/zlib/1.2.11/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/bzip2/1.0.6-CrayGNU-19.08/lib:/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Serialbox/2.4.1-CrayGNU-19.08/lib:/opt/gcc/8.3.0/snos/lib64:/opt/cray/pe/papi/5.7.0.1/lib64:/opt/cray/job/2.2.4-7.0.1.1_3.8__g36b56f4.ari/lib64:/users/omlins/.mujoco/mujoco200/bin (previously undefined)
== 2020-06-04 09:18:00,359 run.py:222 INFO running cmd:  cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_COMPILER='ftn' -DCMAKE_C_FLAGS='-O2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DBOOST_ROOT=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08 -DBoost_NO_SYSTEM_PATHS=ON -DBoost_NO_BOOST_CMAKE=ON -DCMAKE_BUILD_TYPE=Release -DSINGLEPRECISION=OFF -DCUDA_BACKEND=ON -DSTELLA_DIR=${EBROOTSTELLA} -DBITREPRO=OFF  ./../dycore 
== 2020-06-04 09:18:00,548 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/base/exceptions.py:124 in __init__): cmd " cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_COMPILER='ftn' -DCMAKE_C_FLAGS='-O2 -fno-math-errno' -DCMAKE_VERBOSE_MAKEFILE=ON -DBOOST_ROOT=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Boost/1.67.0-CrayGNU-19.08 -DBoost_NO_SYSTEM_PATHS=ON -DBoost_NO_BOOST_CMAKE=ON -DCMAKE_BUILD_TYPE=Release -DSINGLEPRECISION=OFF -DCUDA_BACKEND=ON -DSTELLA_DIR=${EBROOTSTELLA} -DBITREPRO=OFF  ./../dycore" exited with exit code 1 and output:
CMake Error: The source directory "/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/dycore" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
 (at easybuild/software/EasyBuild/4.2.1/lib/python2.7/site-packages/easybuild/tools/run.py:533 in parse_cmd_output)
== 2020-06-04 09:18:00,548 filetools.py:1605 INFO Removing lock /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock...
== 2020-06-04 09:18:00,549 filetools.py:330 INFO Path /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock successfully removed.
== 2020-06-04 09:18:00,549 filetools.py:1609 INFO Lock removed: /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/.locks/_apps_daint_UES_omlins_cosmo_pe1908_keepsrc_eb4.2.1_software_Dycore_551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double.lock
== 2020-06-04 09:18:00,549 easyblock.py:3238 WARNING build failed (first 300 chars): cmd " cmake -DCMAKE_INSTALL_PREFIX=/apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/software/Dycore/551f6b4-CrayGNU-19.08-cuda-10.1-crclim-double -DCMAKE_C_COMPILER='cc' -DCMAKE_Fortran_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_FLAGS='-O2 -fno-math-errno' -DCMAKE_CXX_COMPILER='CC' -DCMAKE_Fortran_C
== 2020-06-04 09:18:00,549 easyblock.py:295 INFO Closing log for application name Dycore version 551f6b4
boegel commented 4 years ago

Since you explicitly had separate_build_dir set to True, which has now become the default, setting it to False isn't going to help of course...

Question remains why it started failing then, because nothing should have changed in that case.

@Flamefire What am I overlooking here?

@omlins Can you share the contents of the /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/ directory (just show the output of ls /apps/daint/UES/omlins/cosmo_pe1908_keepsrc_eb4.2.1/Dycore/551f6b4/)?

Also, your EasyBuild configuration looks a bit weird, it seems like you have buildpath set to /apps, which seems to be the installation prefix? Can you also share the output of eb --show-config?

Flamefire commented 4 years ago

So it looks for /apps/daint/UES/omlins/cosmo_pe1908_keepsrc/Dycore/551f6b4/dycore but you'd expect it to be /apps/daint/UES/omlins/cosmo_pe1908_keepsrc/Dycore/551f6b4/CrayGNU-19.08-cuda-10.1-crclim-double/dycore/? I don't understand where the Cray... part should be coming from.

Usually srcdir is relative to the start_dir (the folder with the extracted sources) or an absolute path which is critical for reproducibility. Also as @boegel mentioned you've set the buildpath and installpath to the same folder. I'd expect problems from this

Anyway my best guess to solve the problem: Check the contents of /apps/common/UES/easybuild/sources/d/%(name)s/%(name)s_%(version)s.tar.gz. Does it contain a CMakeLists.txt? If so remove srcdir from your EC. If that file is in a subfolder set srcdir to the subfolder (name). A .. in srcdir is very likely wrong as you'll be leaving the folder of the extracted sources.

Some notes: