search

xsdk-project / xsdk-issues

A repository under which GitHub issues not related to a specific xSDK repo can be filed.
7 stars 0 forks source link

xsdk-0.6.0 testing on LCF and other machines #124

Closed balos1 closed 3 years ago

balos1 commented 4 years ago

If you have successfully installed the xsdk@0.6.0 on a LCF machine, some other cluster, or a workstation, please leave a comment below with the machine name, the type and the spack command used. For unsuccessful builds, check open issues for the xsdk-0.6.0 release or open a new one.

Things to include in your comment:

Current summary (LCF in bold):

Regular builds

machine name spack arch compiler outstanding issues spack command
summit.olcf.gov linux-rhel7-power9le gcc@7.4.0 #131 spack install xsdk~precice
cori.nersc.gov cray-cnl7-haswell intel@19.0.3.199 none spack install xsdk ^petsc+batch ^boost@1.70.0 ^dealii~oce cflags=-L/opt/cray/pe/atp/2.1.3/libApp cxxflags=-L/opt/cray/pe/atp/2.1.3/libApp
theta.alcf.anl.gov cray-cnl7-mic_knl intel@19.1.0.166 none link
kpro.mcs.anl.gov darwin-catalina-x86_64 apple-clang@12.0.0 none spack install -j6 xsdk%apple-clang@12.0.0 fflags=-fallow-argument-mismatch ^netlib-scalapack cflags=-Wno-implicit-function-declaration
bc.local linux-fedora32-ivybridge gcc@10.2.1 none spack install -j6 xsdk%gcc@10.2.1 ^arpack-ng fflags=-fallow-argument-mismatch ^netlib-lapack
isdp001.cels.anl.gov linux-centos7-mic_knl intel@18.0.2 none python3 ./bin/spack install -j24 xsdk ^suite-sparse@5.3.0 ^boost@1.70.0 ^trilinos target=x86_64
xsdk.icl.utk.edu linux-centos7-skylake_avx512 intel@19.0.3.199 none spack install -j64 xsdk%intel@19.0.3.199~trilinos target=skylake_avx512 ^suite-sparse@5.3.0 ^python%gcc@7.4.0 ^py-numpy%gcc@7.4.0 ^cmake%gcc@7.4.0 ^diffutils%gcc@7.4.0 ^boost@1.70.0
quartz.llnl.gov linux-rhel7-broadwell intel@19.1.2.254 #118, #123 spack install -j48 xsdk~ginkgo~heffte~dealii%intel@19.1.2.254 ^mvapich2 ^matio@1.5.13 ^m4@1.4.17
lassen.llnl.gov linux-rhel7-power9le gcc@7.3.1 #132 spack install -j40 xsdk~dealii%gcc@7.3.1 ^spectrum-mpi ^openblas@0.3.5
pj01.mcs.anl.gov linux-fedora31-skylake clang@9.0.1 none spack install xsdk%clang@9.0.1
si.mcs.anl.gov linux-fedora32-graviton gcc@10.2.1 none spack install xsdk ^arpack-ng fflags=-fallow-argument-mismatch ^pflotran fflags=-fallow-argument-mismatch

CUDA builds

machine name spack arch compiler outstanding issues spack command
summit.olcf.gov linux-rhel7-power9le gcc@7.4.0 #134 spack install xsdk+cuda~trilinos~libensemble~precice~dealii^openblas@0.3.5 ^cuda@10.1.243 ^magma cuda_arch=70
lassen.llnl.gov linux-rhel7-power9le gcc@7.3.1 #132, #135, #136 spack install -j40 xsdk+cuda~dealii %gcc@7.3.1 ^spectrum-mpi@10.3.0.01rtm3 ^openblas@0.3.5 ^magma cuda_arch=70 ^cmake@3.18.2
xsdk.icl.utk.edu linux-centos7-skylake_avx512 gcc@7.4.0 none spack install -j64 xsdk+cuda ^magma cuda_arch=60 ^cuda@10.2.89 ^openblas@0.3.5 ^hwloc@1.11.11
balay commented 4 years ago

kpro.mcs.anl.gov

darwin-catalina-x86_64 / apple-clang@12.0.0

spack install -j6 xsdk%apple-clang@12.0.0 fflags=-fallow-argument-mismatch ^netlib-scalapack cflags=-Wno-implicit-function-declaration

Outstanding issues: none

balay commented 4 years ago

LCF/ cori.nersc.gov

cray-cnl7-haswell / intel@19.0.3.199

spack install xsdk ^petsc+batch ^boost@1.70.0 ^dealii~oce cflags=-L/opt/cray/pe/atp/2.1.3/libApp cxxflags=-L/opt/cray/pe/atp/2.1.3/libApp

Outstanding issues: sundials-5.4.0 failure with MKL #122

balay commented 4 years ago

LCF/ theta.alcf.anl.gov

cray-cnl7-mic_knl / intel@19.1.0.166

balay@thetalogin4:~/spack-jobs> cat spack-build-mon2.sh
#!/bin/sh -x

date
module remove darshan
module remove xalt
module load cce
#module load intel/19.0.5.281
#module load gcc/7.3.0

export HTTPS_PROXY=theta-proxy.tmi.alcf.anl.gov:3128
export https_proxy=theta-proxy.tmi.alcf.anl.gov:3128
export HTTP_PROXY=theta-proxy.tmi.alcf.anl.gov:3128
export http_proxy=theta-proxy.tmi.alcf.anl.gov:3128

aprun -cc none -n 1 python3 /home/balay/spack-xsdk-knl/bin/spack install -j16 xsdk target=mic_knl ^boost@1.70.0

date

Outstanding issues: none

[using some config/fixes in ~/.spack/packages.yaml]

balay commented 4 years ago

bc.local

linux-fedora32-ivybridge / gcc@10.2.1

./bin/spack install  -j6 xsdk%gcc@10.2.1 ^arpack-ng fflags=-fallow-argument-mismatch ^netlib-lapack

Outstanding issues: none

balay commented 4 years ago

isdp001.cels.anl.gov

linux-centos7-mic_knl / intel@18.0.2

python3 ./bin/spack install -j24 xsdk ^suite-sparse@5.3.0 ^boost@1.70.0 ^trilinos target=x86_64

Outstanding issues: sundials-5.4.0 failure with MKL #122

balay commented 4 years ago

xsdk.icl.utk.edu

linux-centos7-skylake_avx512 / gcc@7.4.0

spack install -j64 xsdk+cuda ^magma cuda_arch=60 ^cuda@10.2.89 ^openblas@0.3.5 ^hwloc@1.11.11

Outstanding issues: none

balay commented 4 years ago

xsdk.icl.utk.edu

linux-centos7-skylake_avx512 / intel@19.0.3.199

./bin/spack install -j64 xsdk%intel@19.0.3.199~trilinos target=skylake_avx512 ^suite-sparse@5.3.0 ^python%gcc@7.4.0 ^py-numpy%gcc@7.4.0 ^cmake%gcc@7.4.0 ^diffutils%gcc@7.4.0 ^boost@1.70.0

Outstanding issues: none

balos1 commented 4 years ago

quartz.llnl.gov

linux-rhel7-broadwell / intel@19.1.2.254

spack install -j48 xsdk~ginkgo~heffte~dealii%intel@19.1.2.254 ^mvapich2 ^matio@1.5.13 ^m4@1.4.17

Outstanding issues: ginkgo issue with intel compilers linked to an outdated gcc (looks the only workaround is to use a different compiler or disable ginkgo) #118, heFFTe issue with intel compilers (seems to be a compiler problem) #123.

luszczek commented 4 years ago

summit.olcf.gov

linux-rhel7-power9le/gcc-6.4.0

Outstanding issues:

  1. butterflypack error due to OpenMP in dBplus_utilities.f90 https://github.com/xsdk-project/xsdk-issues/issues/127
  2. deal.II requires Intel's TBB. https://github.com/xsdk-project/xsdk-issues/issues/128
liuyangzhuan commented 4 years ago

This is strange, I have guided all taskloop syntax with "#ifdef HAVE_TASKLOOP" which is defined by CMAKE automatically.

Can you send the complete build log?

balay commented 4 years ago

@luszczek @liuyangzhuan its best to create a different issue on the issue tracker for the summit issues [and add logs there - and make progresss on the issue]. And add a reference here. Ok - will create the new issues now - and update this issue.

balos1 commented 4 years ago

lassen.llnl.gov

linux-rhel7-power9le / gcc@7.3.1

spack install -j40 xsdk~dealii%gcc@7.3.1 ^spectrum-mpi ^openblas@0.3.5

Outstanding issues: #132

luszczek commented 4 years ago

summit.olcf.gov

linux-rhel7-power9le / gcc@7.4.0

spack install xsdk~precice

But this had #131 patched

luszczek commented 4 years ago

summit.olcf.ornl.gov

linux-rhel7-power9le / gcc@7.4.0

Succeeded with:

install xsdk+cuda~trilinos~libensemble~precice^openblas@0.3.5 ^magma cuda_arch=70

But I had to disable mfem due to issue #134

luszczek commented 4 years ago
balay commented 4 years ago
balay commented 4 years ago
balos1 commented 4 years ago

lassen.llnl.gov

linux-rhel7-power9le / gcc@7.3.1

spack install -j40 xsdk+cuda~dealii %gcc@7.3.1 ^spectrum-mpi@10.3.0.01rtm3  ^openblas@0.3.5 ^magma cuda_arch=70 ^cmake@3.18.2

Pending Issues:

132 (this is why ~dealii), #135 , #136 (using cmake 3.18.2 is a workaround for the latter 2 issues).