C2SM / spack-c2sm

Repository for c2sm spack config and repo files
https://c2sm.github.io/spack-c2sm/latest
MIT License
7 stars 19 forks source link

spack installcosmo --only dependencies returns 0 on failure #236

Closed cosunae closed 3 years ago

cosunae commented 3 years ago

spack installcosmo cosmo cmd returns properly error when it fails, but while building only the dependencies, it does not returns a failure.

To reproduce it:

spack installcosmo -j 20 --only dependencies cosmo@5.08.mch1.0.p3%pgi@20.4 +set_version  real_type=float cosmo_target=gpu +production +claw +verbose +pollen +eccodes

if any of the packages fail, the command still returns 0. We should make it return an error

BenWeber42 commented 3 years ago

I investigated this and noticed that spack install behaves the same (which makes sense since spack installcosmo copies much of the code from spack install ). This means that this feature isn't in line with existing features in spack. So for me it's unclear if we still want to do this. If we really want this, I probably could find a way to get it done, but the hard part will be to make sure that it won't break if we update the spack version.

Here the log that shows that spack install behaves the same:

$ spack install --only=dependencies cosmo@5.08.mch1.0.p3%pgi@20.4 +set_version  real_type=float cosmo_target=gpu +production +claw +verbose +pollen +eccodes
==> Warning: Missing a source id for openjdk@1.8.0
==> Warning: Missing a source id for flex@2.5.37
==> Warning: Missing a source id for slurm@19.05.05
==> openjdk@1.8.0 : externally installed in /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64
[+] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64
==> bison@3.0.4 : externally installed in /usr
[+] /usr
==> cmake@3.14.5 : has external module in cmake/3.14.5
[+] /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/generic/easybuild/software/CMake/3.14.5
==> flex@2.5.37 : externally installed in /usr
[+] /usr
==> libxml2@2.9.9 : externally installed in /usr
[+] /usr
==> bzip2@1.0.6 : externally installed in /usr
[+] /usr
==> zlib@1.2.11 : externally installed in /usr
[+] /usr
==> cuda@10.1.243 : has external module in cuda/10.1.243
[+] /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/generic/easybuild/software/CUDA/10.1.243
==> openmpi@4.0.2 : has external module in /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/pgi/20.4/easybuild/modules/all/openmpi/4.0.2-pgi-20.4-gcc-8.3.0-cuda-10.1
[+] /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/pgi/20.4/easybuild/software/OpenMPI/4.0.2-PGI-20.4-GCC-8.3.0-cuda-10.1
==> slurm@19.05.05 : has external module in slurm/19.05.05
[+] None
[+] /project/g110/spack-install/tsa/nasm/2.14.02/gcc/5fvyet37iloe24yhbk32gszjpi5bysts
[+] /project/g110/spack-install/tsa/nasm/2.14.02/gcc/5fvyet37iloe24yhbk32gszjpi5bysts
[+] /project/g110/spack-install/tsa/libgrib1/22-01-2020/pgi/2gwunsmov5q6myzwvlukjzlrupljnnhv
[+] /project/g110/spack-install/tsa/libgrib1/22-01-2020/pgi/2gwunsmov5q6myzwvlukjzlrupljnnhv
==> netcdf-c@4.7.0 : has external module in /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/pgi/20.4/easybuild/modules/all/netcdf/4.7.0-pgi-20.4-gcc-8.3.0
[+] /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/pgi/20.4/easybuild/software/netCDF/4.7.0-PGI-20.4-GCC-8.3.0
==> netcdf-fortran@4.4.5 : has external module in /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/pgi/20.4/easybuild/modules/all/netcdf-fortran/4.4.5-pgi-20.4-gcc-8.3.0
[+] /apps/arolla/UES/jenkins/RH7.9/MCH-PE20.08-UP01/pgi/20.4/easybuild/software/netCDF-Fortran/4.4.5-PGI-20.4-GCC-8.3.0
[+] /project/g110/spack-install/tsa/omni-xmod-pool/0.1/pgi/fa6l6gthqbp7gqxybl66yd5l7qrfhgjf
[+] /project/g110/spack-install/tsa/omni-xmod-pool/0.1/pgi/fa6l6gthqbp7gqxybl66yd5l7qrfhgjf
==> perl@5.16.3 : externally installed in /usr
[+] /usr
[+] /project/g110/spack-install/tsa/ant/1.10.7/gcc/spehxcfrpnhss7y62ozbgc66icoxhvoy
[+] /project/g110/spack-install/tsa/ant/1.10.7/gcc/spehxcfrpnhss7y62ozbgc66icoxhvoy
[+] /scratch/wbenjami/spack-install/tsa/boost/1.67.0/gcc/u3fxoae43srcjebtiwgj4jf73zmpch2o
[+] /project/g110/spack-install/tsa/libjpeg-turbo/2.0.4/gcc/maf3jzml7ivt4kvgj2r2u2ljhluqhgum
[+] /project/g110/spack-install/tsa/libjpeg-turbo/2.0.4/gcc/maf3jzml7ivt4kvgj2r2u2ljhluqhgum
[+] /project/g110/spack-install/tsa/claw/2.1/gcc/cbuuxucbgruzvz7wvtzm22ukkdrupgfj
[+] /project/g110/spack-install/tsa/claw/2.1/gcc/cbuuxucbgruzvz7wvtzm22ukkdrupgfj
[+] /scratch/wbenjami/spack-install/tsa/serialbox/2.6.0/gcc/3u63amjacjzjco2n62z5neoet2gnvkrx
[+] /project/g110/spack-install/tsa/jasper/1.900.1/gcc/wqqmp5apkmtlgrwpr2rdms654fcflm3k
[+] /project/g110/spack-install/tsa/jasper/1.900.1/gcc/wqqmp5apkmtlgrwpr2rdms654fcflm3k
==> Installing cosmo-dycore
==> No binary for cosmo-dycore found: installing from source
Fetching tags only, you probably meant:
  git fetch --tags
==> cosmo-dycore: Executing phase: 'cmake'
==> Error: InstallError: Artificially making this fail!

/users/wbenjami/.spack/repo/packages/cosmo-dycore/package.py:106, in cmake_args:
        104    def cmake_args(self):
        105
  >>    106      raise InstallError("Artificially making this fail!")
        107
        108      spec = self.spec
        109

See build log for details:
  /scratch/wbenjami/spack-stages/tsa/spack-stage-cosmo-dycore-5.09a.mch1.1.p1-bwhswwzxz4pqnzyiu46smjfxgb7fphgg/spack-build-out.txt

[+] /project/g110/spack-install/tsa/eccodes/2.19.0/pgi/jrtkwxeou7z7ogayybcxvanytorkc5yl
[+] /project/g110/spack-install/tsa/eccodes/2.19.0/pgi/jrtkwxeou7z7ogayybcxvanytorkc5yl
[+] /project/g110/spack-install/tsa/cosmo-eccodes-definitions/2.19.0.5/pgi/xr7ba6lqw35o73bjm2pc5kgq3w7hbywx
[+] /project/g110/spack-install/tsa/cosmo-eccodes-definitions/2.19.0.5/pgi/xr7ba6lqw35o73bjm2pc5kgq3w7hbywx
$ echo $?
0