search

NOAA-EMC / WW3

WAVEWATCH III
Other
265 stars 542 forks source link

netCDF include error #426

Closed sbrus89 closed 7 months ago

sbrus89 commented 3 years ago

Describe the bug I got an error compiling netCDF programs due to a suspected issue with netCDF include directories.

To Reproduce Steps to reproduce the behavior:

cd WW3/model/bin
./w3_setup ../../model -c intel -s NCEP_glwu
export WWATCH3_NETCDF="NC4"
export NETCDF_CONFIG="/example/path/intel-17.0.0/netcdf-4.4.1-tckdgwl/bin/nc-config"
./w3_make ww3_ounf

Additional context https://github.com/NOAA-EMC/WW3/blob/3c4732490b8d1d4e1ca2af81e71d8cd30c1de950/model/bin/comp.tmpl#L82-L84 I believe that switching --cflags to --fflags will fix this issue. This makes sure that the fortran-netcdf headers are included.

JessicaMeixner-NOAA commented 3 years ago

What exactly is the error? I believe we intentionally changed this to --cflags (and --fflags didn't work) but I also think @mickaelaccensi has some problems with this too. Its possible this is an issue with how netCDF is installed. For @mickaelaccensi his issues I want to say were resolved when switching from using nf-config to nc-config.

sbrus89 commented 3 years ago

@JessicaMeixner-NOAA, the errors are:

w3ounfmetamd.F90(164): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [NETCDF]
      USE NETCDF
----------^
w3ounfmetamd.F90(1870): error #6404: This name does not have a type, and must have an explicit type.   [NF90_PUT_ATT]
      ERR = NF90_PUT_ATT(NCID, VARID, 'long_name', META%VARNL)
------------^
w3ounfmetamd.F90(1871): error #6404: This name does not have a type, and must have an explicit type.   [NF90_NOERR]
      IF(ERR /= NF90_NOERR) RETURN
----------------^
w3ounfmetamd.F90(1888): error #6404: This name does not have a type, and must have an explicit type.   [NF90_FILL_SHORT]
        ERR = NF90_PUT_ATT(NCID, VARID, '_FillValue', NF90_FILL_SHORT)
------------------------------------------------------^
w3ounfmetamd.F90(1890): error #6404: This name does not have a type, and must have an explicit type.   [NF90_FILL_FLOAT]
        ERR = NF90_PUT_ATT(NCID, VARID, '_FillValue', NF90_FILL_FLOAT)
------------------------------------------------------^
w3ounfmetamd.F90(1982): error #6404: This name does not have a type, and must have an explicit type.   [NF90_GLOBAL]
      CALL WRITE_FREEFORM_META_LIST(NCID, NF90_GLOBAL, GLOBAL_META, ERR)
------------------------------------------^
w3ounfmetamd.F90(2042): error #6404: This name does not have a type, and must have an explicit type.   [NF90_PUT_ATT]
            ERR = NF90_PUT_ATT(NCID, VARID, P%ATTNAME, IVAL)
------------------^
compilation aborted for w3ounfmetamd.F90 (code 1)

Good to know about --fflags not working. That's what fixed it for me, but obviously it's machine-dependent. You're probably right that this has something to do with how netCDF is installed.

JessicaMeixner-NOAA commented 3 years ago

Probably need to think of a general way to solve this. I think @mickaelaccensi had an idea, he's also on vacation for the next week or two.

sbrus89 commented 3 years ago

No problem; it'd be interesting to hear what he has in mind when he's back.

It's unfortunate that the current strategy isn't as general as it should be, because it's really a nice way to handle this. It seems like generality is exactly the thing that --cflags and --fflags are designed to achieve.

mickaelaccensi commented 3 years ago

@sbrus89, could you provide the output of the commands : 

 nc-config --all
 nf-config --all

I'll try to add a patch to detect this error, on my side, the problem was on the link stage because the option '--libs' does not exist for nf-config.

sbrus89 commented 3 years ago

@mickaelaccensi, here the output of nc-config --all:


This netCDF 4.4.1 has been built with the following features: 

  --cc        -> /blues/gpfs/software/centos7/spack-latest/lib/spack/env/intel/icc
  --cflags    ->  -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-4.4.1-tckdgwl/include -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/curl-7.63.0-m6myl33/include -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/hdf5-1.8.16-bkyfrve/include -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/hdf-4.2.13-3cgkqy6/include
  --libs      -> 

  --has-c++   -> no
  --cxx       -> 
  --has-c++4  -> no
  --cxx4      -> 

  --fc        -> /blues/gpfs/software/centos7/spack-latest/lib/spack/env/intel/ifort
  --fflags    -> -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss/include
  --flibs     -> -L/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss/lib -lnetcdff -L/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-4.4.1-tckdgwl/lib -lnetcdf -lnetcdf
  --has-f90   -> no
  --has-f03   -> yes

  --has-dap   -> yes
  --has-nc2   -> yes
  --has-nc4   -> yes
  --has-hdf5  -> yes
  --has-hdf4  -> yes
  --has-logging-> no
  --has-pnetcdf-> no
  --has-szlib -> 

  --prefix    -> /blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-4.4.1-tckdgwl
  --includedir-> /blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-4.4.1-tckdgwl/include
  --version   -> netCDF 4.4.1
sbrus89 commented 3 years ago

and here's the output of nf-config --all:

This netCDF-Fortran 4.4.4 has been built with the following features: 

  --cc        -> /blues/gpfs/software/centos7/spack-latest/lib/spack/env/intel/icc
  --cflags    ->  -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss/include -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-4.4.1-tckdgwl/include

  --fc        -> /blues/gpfs/software/centos7/spack-latest/lib/spack/env/intel/ifort
  --fflags    -> -I/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss/include
  --flibs     -> -L/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss/lib -lnetcdff -L/blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-4.4.1-tckdgwl/lib -lnetcdf -lnetcdf 
  --has-f90   -> no
  --has-f03   -> yes

  --has-nc2   -> yes
  --has-nc4   -> yes

  --prefix    -> /blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss
  --includedir-> /blues/gpfs/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/intel-17.0.0/netcdf-fortran-4.4.4-urmb6ss/include
  --version   -> netCDF-Fortran 4.4.4
sbrus89 commented 3 years ago

Let me know what other information I can provide. Thanks for your help with this!

mickaelaccensi commented 3 years ago

@JessicaMeixner-NOAA, @thesser1 and @ukmo-ccbunney , could you also provide the output of :

nc-config --all
nf-config --all

The idea is to implement a smart patch for the comp and link options for netcdf in ww3.

From my point of view, there are 2 problems : -if the netcdf-c and netcdf-fortran are not compiled in the same directory ->that's the case of Steven, fflags and cflags provide different results for both nc-config and nf-config -if nf-config is set for $NETCDF_CONFIG, then the option --libs does not exist -> that's the case for all of us, the compilation crash so an explicit error message should be added

ukmo-ccbunney commented 3 years ago

@mickaelaccensi Here is the output from nc-config --all on our HPC:

This netCDF 4.6.1 has been built with the following features: 

  --cc        -> cc
  --cflags    -> -DpgiFortran
  --libs      -> 

  --has-c++   -> no
  --cxx       -> 

  --has-c++4  -> yes
  --cxx4      -> CC
  --cxx4flags -> -DpgiFortran
  --cxx4libs  -> 

  --has-fortran-> yes
  --fc        -> ftn
  --fflags    -> 
  --flibs     -> 
  --has-f90   -> no
  --has-f03   -> yes

  --has-dap   -> no
  --has-dap4  -> no
  --has-nc2   -> yes
  --has-nc4   -> yes
  --has-hdf5  -> yes
  --has-hdf4  -> no
  --has-logging-> no
  --has-pnetcdf-> no
  --has-szlib -> no
  --has-parallel -> no
  --has-cdf5 -> yes

  --prefix    -> /opt/cray/netcdf/4.6.1.3/gnu/7.1
  --includedir-> /opt/cray/netcdf/4.6.1.3/include
  --libdir    -> /opt/cray/netcdf/4.6.1.3/gnu/7.1/lib
  --version   -> netCDF 4.6.1

and the same for nf-config --all:

This netCDF-Fortran 4.4.4 has been built with the following features: 

  --cc        -> cc
  --cflags    -> -DpgiFortran

  --fc        -> ftn
  --fflags    -> 
  --flibs     -> 
  --has-f90   -> no
  --has-f03   -> yes

  --has-nc2   -> yes
  --has-nc4   -> yes

  --prefix    -> /opt/cray/netcdf/4.6.1.3/gnu/7.1
  --includedir-> /opt/cray/netcdf/4.6.1.3/gnu/7.1/include
  --version   -> netCDF-Fortran 4.4.4
JessicaMeixner-NOAA commented 3 years ago

here's for nc-config --all: 


This netCDF 4.7.4 has been built with the following features: 

  --cc            -> mpiicc
  --cflags        -> -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include  -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/hdf5/1.10.6/include
  --libs          -> -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/hdf5/1.10.6/lib -lhdf5_hl -lhdf5   -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/zlib/1.2.11/lib  -lz -ldl -lm   -lnetcdf -lhdf5_hl -lhdf5 -lm -lz 
  --static        -> -lhdf5_hl -lhdf5 -lm -lz 

  --has-c++       -> no
  --cxx           -> 

  --has-c++4      -> yes
  --cxx4          -> mpiicpc
  --cxx4flags     -> -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/hdf5/1.10.6/include
  --cxx4libs      -> -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/hdf5/1.10.6/lib -lhdf5_hl -lhdf5 -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/zlib/1.2.11/lib -lz -ldl -lm -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib -lnetcdf -lnetcdf_c++4 -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/hdf5/1.10.6/lib -lhdf5_hl -lhdf5 -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/zlib/1.2.11/lib -lz -ldl -lm -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lhdf5_hl -lhdf5 -lm -lz -lnetcdf

  --has-fortran   -> yes
  --fc            -> mpiifort
  --fflags        -> -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include
  --flibs         -> -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib -lnetcdff
  --has-f90       -> 
  --has-f03       -> yes

  --has-dap       -> no
  --has-dap2      -> no
  --has-dap4      -> no
  --has-nc2       -> yes
  --has-nc4       -> yes
  --has-hdf5      -> yes
  --has-hdf4      -> no
  --has-logging   -> no
  --has-pnetcdf   -> no
  --has-szlib     -> no
  --has-cdf5      -> yes
  --has-parallel4 -> yes
  --has-parallel  -> yes

  --prefix        -> /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4
  --includedir    -> /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include
  --libdir        -> /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib
  --version       -> netCDF 4.7.4

and for nf-config --all


This netCDF-Fortran 4.5.3 has been built with the following features: 

  --cc        -> mpiicc
  --cflags    -> -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include  -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/hdf5/1.10.6/include

  --fc        -> mpiifort
  --fflags    -> -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include
  --flibs     -> -L/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/lib -lnetcdff 
  --has-f90   -> 
  --has-f03   -> yes

  --has-nc2   -> yes
  --has-nc4   -> yes

  --prefix    -> /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4
  --includedir-> /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include
  --version   -> netCDF-Fortran 4.5.3
JessicaMeixner-NOAA commented 2 years ago

@sbrus89 I lost track of this issue. Is this still a problem? Since we have a pretty big build system change coming up with PR #533, I'm wondering if we should revisit this after that? Thoughts?

sbrus89 commented 2 years ago

@JessicaMeixner-NOAA, I think that's a good idea to revisit after #533. It could be that the CMake PR will take care of this or make solving it more straightforward. I still run into this type of thing every so often when compiling on new systems, but I seem to always be able to make adjustments to get things working.

mickaelaccensi commented 7 months ago

now it is solved by using cmake, so I close this issue. If you still encountered issue with compilation using cmake, please open a new issue

best,