search

xsdk-project / xsdk-issues

A repository under which GitHub issues not related to a specific xSDK repo can be filed.
7 stars 0 forks source link

PFLOTRAN 4.0.1 build issue with Cray's Fortran compiler #204

Closed v-dobrev closed 1 year ago

v-dobrev commented 2 years ago

The Spack spec of the failing build is:

pflotran@4.0.1%cce@14.0.4 build_system=autotools arch=linux-rhel8-zen3/yn37hbd

The relevant lines from the log are as follows:

...

/opt/cray/pe/mpich/8.1.20/ofi/crayclang/10.0/bin/mpif90 -o material_aux.o -c -fPIC    -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/petsc-3.18.1-yqut5abl7jexpmjmuvxotnc55oqsb3fh/include -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/hypre-2.26.0-kltrgzkoyrofm4okh3fcwf5zxb6bx5mj/include -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/superlu-dist-8.1.2-wviaio2oafj7iqeplwqsva76bwp273pf/include -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/hdf5-1.12.2-2jz4x26eywipd5ghyitgmzjze6wlc2ok/include -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/parmetis-4.0.3-j62a2tvdw4rfv5qt2npuk45vecbkd5ea/include -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/metis-5.1.0-omwrvbhhtxtnecgmd2mpggfjwikowt7y/include -I/usr/WS2/mfem/spack-tioga/opt/spack/linux-rhel8-zen3/cce-14.0.4/zlib-1.2.13-6me6bkiccxpyxnovgm24vqm3hkizim54/include -I/opt/rocm-5.2.3/include   -I. `pwd`/material_aux.F90

...

ftn-855 ftn: ERROR MATERIAL_AUX_CLASS, File = ../../spack-stage/spack-stage-pflotran-4.0.1-yn37hbduckq6onj3xedxplrveqmbmhei/spack-src/src/pflotran/material_aux.F90, Line = 1, Column = 8 
  The compiler has detected errors in module "MATERIAL_AUX_CLASS".  No module information file will be created for this module.

ftn-355 ftn: ERROR MATERIALAUXVARCOMPUTE, File = ../../spack-stage/spack-stage-pflotran-4.0.1-yn37hbduckq6onj3xedxplrveqmbmhei/spack-src/src/pflotran/material_aux.F90, Line = 746, Column = 10 
  Generic interface "MATERIALCOMPRESSSOIL" is referenced in a declarative expression in a [sub]module.  Specific "MATERIALCOMPRESSSOILPTR" must be completely specified prior to being referenced.

ftn-389 ftn: ERROR MATERIALAUXVARCOMPUTE, File = ../../spack-stage/spack-stage-pflotran-4.0.1-yn37hbduckq6onj3xedxplrveqmbmhei/spack-src/src/pflotran/material_aux.F90, Line = 746, Column = 10 
  No specific match can be found for the generic subprogram call "MATERIALCOMPRESSSOIL".

Cray Fortran : Version 14.0.4 (20220914025807_3d8a48c51d4c92570b90f8f94df80601b08918b8)
Cray Fortran : Compile time:  0.0576 seconds
Cray Fortran : 1757 source lines
Cray Fortran : 3 errors, 0 warnings, 0 other messages, 0 ansi
Cray Fortran : "explain ftn-message number" gives more information about each message.
balay commented 2 years ago

cc: @ghammond86

ghammond86 commented 2 years ago

Using NERSC compilers, I fixed the "error" above and several subsequent errors reported by the Cray Fortran compiler. Although fixing several of these will result in cleaner code, many are baseless. I believe the one above to be baseless; it actually degrades modularity. There is also a cryptic link error that makes no sense, for which i cannot find a workaround.

/opt/cray/pe/cce/14.0.1/binutils/x86_64/x86_64-pc-linux-gnu/bin/ld: inversion_subsurface.o:(.bss+0x40): multiple definition of `$host_init$inversioncoupledsolutioninit$inversion_coupled_auxmodule'; inversion_coupled_aux.o:(.bss+0x0): first defined here

When we refactored PFLOTRAN to incorporated Fortran classes (~2013 timeframe), we gave up on the Cray and PGI compilers because they were either buggy, failed to support critical features in the F03/08 standards, or reported misleading/bogus issues. Compilation with PGI was also prohibitively slow.

My recommendation is to not compile PFLOTRAN through PETSc for Cray. The reaction capability needed by Amanzi-ATS is provided through a reaction library that is a subset of PFLOTRAN. That library should compile without issues.

v-dobrev commented 2 years ago

My recommendation is to not compile PFLOTRAN through PETSc for Cray. The reaction capability needed by Amanzi-ATS is provided through a reaction library that is a subset of PFLOTRAN. That library should compile without issues.

What will be the change in the xSDK and/or PFLOTRAN Spack recipes to achieve this?

ghammond86 commented 2 years ago

My recommendation is to not compile PFLOTRAN through PETSc for Cray. The reaction capability needed by Amanzi-ATS is provided through a reaction library that is a subset of PFLOTRAN. That library should compile without issues.

What will be the change in the xSDK and/or PFLOTRAN Spack recipes to achieve this?

@balay I recommend building pflotran ("make pflotran") for all compilers other than Cray and building pflotran_rxn ("make pflotran_rxn") for Cray. I confirmed that pflotran_rxn builds with Cray compilers on NERSC perlmutter.

balay commented 2 years ago

My recommendation is to not compile PFLOTRAN through PETSc for Cray

Do I understand correctly that pflotran_rxn doesn't require PETSc?

I'm not sure who to implement depends_on("petsc", when="not using cray-compilers"). Perhaps @v-dobrev knows how to encode this.

Alternative is to add a variant rxn.

I have this [untested] change at https://github.com/spack/spack/commits/balay/pflotran-rxn

balay commented 2 years ago

Do I understand correctly that pflotran_rxn doesn't require PETSc?

==> [2022-11-17-11:29:04.206829] '/home/balay/spack.orig/spack-stage/spack-stage-pflotran-4.0.1-4yetyyiywki77yhxtfqorwyrjbdj6usm/spack-src/configure' '--prefix=/home/balay/spack.orig/opt/spack/linux-fedora37-skylake/oneapi-2022.2.0/pflotran-4.0.1-4yetyyiywki77yhxtfqorwyrjbdj6usm'
Need to set PETSC_DIR

Ok - I guess I mis-understood.

balay commented 2 years ago

I reset the petsc dependency. Also make pflotran_rxn is in src/pflotran - and not top-level makefile.

And I pushed my changes.

A basic build of pflotran+rxn works on my linux box. Perhaps @v-dobrev can try a build with this branch.

BTW: alternative (that might work): spack install xsdk@cce... ^pflotran@gcc...

ghammond86 commented 2 years ago

Do I understand correctly that pflotran_rxn doesn't require PETSc?

pflotran_rxn is a simple reaction wrapper code that links to libpflotranchem.a, a subset library that covers the chemical reaction capability. "Everything pflotran" requires PETSc. The current Cray bugs/issues/incompatibilities within pflotran are outside the libpflotranchem.a subset of the code.

libpflotranchem.a is the library leveraged by Amanzi-ATS. I note that Amanzi-ATS does not appear to use xsdk to access libpflotranchem.a. It builds it separately.

I'm not sure who to implement depends_on("petsc", when="not using cray-compilers"). Perhaps @v-dobrev knows how to encode this.

Alternative is to add a variant rxn.

I have this [untested] change at https://github.com/spack/spack/commits/balay/pflotran-rxn

balay commented 2 years ago

A basic build of pflotran+rxn works on my linux box. Perhaps @v-dobrev can try a build with this branch.

PR at: https://github.com/spack/spack/pull/33995

v-dobrev commented 2 years ago

I see that the Spack PR is now merged -- I'll try to test it later today in spack@develop.

balay commented 2 years ago

I get: (on crusher)

/opt/cray/pe/mpich/8.1.19/ofi/crayclang/10.0/bin/mpif90 -o material_aux.o -c -fPIC    -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/petsc-3.18.1-tsogulp4i2rbkv5dandoqujue2nillch/include -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/hypre-2.26.0-4446f45xmz2y5nulhwt3zqrkwylm37uh/include -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/superlu-dist-8.1.2-ei2bxwc63rh67kwqamcexdhch3t35hjv/include -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/hdf5-1.12.2-bbryfrqis5vf6oni3ksyqj575m6cqttu/include -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/parmetis-4.0.3-z6ccmufyde25spgphr2xfpfigmzlvuop/include -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/metis-5.1.0-q73tcdvoto3jjuimj6z5bbxemotzghdw/include -I/tmp/balay/spack.x/opt/spack/linux-sles15-zen3/cce-14.0.3/zlib-1.2.13-fjcnr3uycpvayywf4mk2trbtypwkpdnu/include -I/opt/rocm-5.2.0/include   -I. `pwd`/material_aux.F90
ftn-855 ftn: ERROR MATERIAL_AUX_CLASS, File = ../../balay/spack-stage/spack-stage-pflotran-4.0.1-vnwd4uki32fjq7eyvnmmxr34d7glt324/spack-src/src/pflotran/material_aux.F90, Line = 1, Column = 8
  The compiler has detected errors in module "MATERIAL_AUX_CLASS".  No module information file will be created for this module.

ftn-355 ftn: ERROR MATERIALAUXVARCOMPUTE, File = ../../balay/spack-stage/spack-stage-pflotran-4.0.1-vnwd4uki32fjq7eyvnmmxr34d7glt324/spack-src/src/pflotran/material_aux.F90, Line = 746, Column = 10
  Generic interface "MATERIALCOMPRESSSOIL" is referenced in a declarative expression in a [sub]module.  Specific "MATERIALCOMPRESSSOILPTR" must be completely specified prior to being referenced.

ftn-389 ftn: ERROR MATERIALAUXVARCOMPUTE, File = ../../balay/spack-stage/spack-stage-pflotran-4.0.1-vnwd4uki32fjq7eyvnmmxr34d7glt324/spack-src/src/pflotran/material_aux.F90, Line = 746, Column = 10
  No specific match can be found for the generic subprogram call "MATERIALCOMPRESSSOIL".

Cray Fortran : Version 14.0.3 (20220818062825_80f97ce63a7725610a864116e454efffd075b025)
Cray Fortran : Compile time:  0.0507 seconds
Cray Fortran : 1753 source lines
Cray Fortran : 3 errors, 0 warnings, 0 other messages, 0 ansi
Cray Fortran : "explain ftn-message number" gives more information about each message.
v-dobrev commented 1 year ago

I just tried building xsdk@0.8.0 with ^pflotran+rxn on Tioga and got the same error as on Crusher.