[INSTALL] ESMF on linux issue

danielabdi-noaa commented 1 year ago

Describe the bug ESMF with gnu+openmpi does not build on linux

To Reproduce Steps to reproduce the behavior:

esmf:
  build: YES 
  version: v8.3.0b09
  install_as: 8.3.0b09
  shared: YES 
  enable_pnetcdf: NO
  debug: NO

./build_stack.sh -p /home/daniel/apps/hpc-stack -c /home/daniel/srw-linux/hpc-stack/config/config_custom.sh -y stack/my_stack.yaml -m

mpifort -c -g -fbacktrace   -fPIC -O2 -fPIC  -m64 -mcmodel=small -pthread -ffree-line-length-none  -fopenmp -I/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO -I/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/build_config/Linux.gfortran.default -I/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure -I/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Superstructure -I/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/mod/modO/Linux.gfortran.64.openmpi.default -I/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/include -I/home/daniel/apps/hpc-stack/gnu-10.3.0/openmpi-4.0.3/netcdf/4.7.4/include  -D_NETCDF -DNO_MPIMOD -DESMF_NO_INTEGER_1_BYTE -DESMF_NO_INTEGER_2_BYTE -DESMF_VERSION_STRING_GIT='"v8.3.0b09"' -DESMF_MOAB=1 -DESMF_LAPACK=1 -DESMF_LAPACK_INTERNAL=1 -DESMF_NO_ACC_SOFTWARE_STACK=1 -DESMF_NETCDF=1 -DESMF_YAMLCPP=1 -DESMF_YAML=1 -DESMF_PIO=1 -DESMF_MPIIO -DESMF_NO_OPENACC -DESMF_BOPT_O -DESMF_TESTCOMPTUNNEL -DSx86_64_small=1 -DESMF_OS_Linux=1 -DESMF_COMM=openmpi -DESMF_DIR=/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09 /home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO/pio_support.F90 -o /home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/obj/objO/Linux.gfortran.64.openmpi.default/pio_support.o
/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO/pio_support.F90:331:27:

  331 |              call MPI_SEND(sdof,1,PIO_OFFSET_KIND,n,n,comm,ierr)
      |                           1
......
  496 |                   call mpi_send ( hs, 1, MPI_INTEGER, p, mtag, comm, ier )
      |                                  2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/INTEGER(4)).
/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO/pio_support.F90:334:30:

  334 |                 call MPI_SEND(wdof,int(sdof),PIO_OFFSET_KIND,n,npes+n,comm,ierr)
      |                              1
......
  496 |                   call mpi_send ( hs, 1, MPI_INTEGER, p, mtag, comm, ier )
      |                                  2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/INTEGER(4)).
/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO/pio_support.F90:342:21:

  342 |        call MPI_RECV(sdof,1,PIO_OFFSET_KIND,masterproc,myrank,comm,status,ierr)
      |                     1
......
  515 |             call mpi_recv  ( hs, 1, MPI_INTEGER, root, mtag, comm, &
      |                             2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/INTEGER(4)).
/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO/pio_support.F90:347:24:

  347 |           call MPI_RECV(dof,int(sdof),PIO_OFFSET_KIND,masterproc,npes+myrank,comm,status,ierr)
      |                        1
......
  515 |             call mpi_recv  ( hs, 1, MPI_INTEGER, root, mtag, comm, &
      |                             2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/INTEGER(4)).
/home/daniel/srw-linux/hpc-stack/pkg/v8.3.0b09/src/Infrastructure/IO/PIO/pio_support.F90:222:27:

  222 |              call MPI_SEND(dof,int(sdof),PIO_OFFSET_KIND,masterproc,n,comm,ierr)
      |                           1
......
  496 |                   call mpi_send ( hs, 1, MPI_INTEGER, p, mtag, comm, ier )
      |                                  2

Expected behavior ESMF should build. I noticed that even though enable_pnetcdf: NO, it is trying to build this particular file with -DESMF_PIO=1 -DESMF_MPIIO

System: Linux Ubuntu 20.04

Additional context Add any other context about the problem here. gnu/10.3.0 and openmpi/4.0.3 python/3.9.5 all come from my system.

climbfuji commented 1 year ago

You need the "correct compiler flags" (i.e. the duct-tape solution): https://github.com/NOAA-EMC/spack/blob/a5127b4b834ffcb524dd7b34bed496a791bad962/var/spack/repos/builtin/packages/esmf/package.py#L240 - I think hpc-stack has an example in one of its site configs for how to set it.

danielabdi-noaa commented 1 year ago

@climbfuji Thanks! I am able to build esmf now using that hack.

NOAA-EMC / hpc-stack

[INSTALL] ESMF on linux issue #508