search

wrf-model / WRF

The official repository for the Weather Research and Forecasting (WRF) model
Other
1.25k stars 691 forks source link

Error compiling WRF Plus in WRF 4.0.2 on module_wrf_top.f90 #709

Closed cross85 closed 5 years ago

cross85 commented 5 years ago

I keep getting an error message when I try to compile WRF Plus. I configured with intel compilers (option 10) and netcdf 4.6.2 (without netcdf4), and I have no problem when I compiled WRF and WPS. So, when I compile WRF Plus I get the following error messages:

mpiifort -o ../main/module_wrf_top.o -c -O3 -xCORE-AVX512 -w  -auto -ftz -fno-alias -fp-model fast=1 -no-prec-div -no-prec-sqrt -FR -convert big_endian -auto -align array64byte    -I../dyn_em -I../dyn_nmm  -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/external/esmf_time_f90  -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/main -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/external/io_netcdf -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/external/io_int -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/frame -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/share -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/phys -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/wrftladj -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/chem -I/home/cross/WRF/4.0.2_nc3/WRF_Plus/inc -I/home/cross/Software/nc3/libs/netcdf/include  -r8        -real-size `expr 8 \* 8` -i4  ../main/module_wrf_top.f90
../main/module_wrf_top.f90(756): error #6099: An ENDDO statement occurred without a corresponding DO or DO WHILE statement.
   ENDDO
^
compilation aborted for ../main/module_wrf_top.f90 (code 1)
../configure.wrf:346: recipe for target '../main/module_wrf_top.o' failed

So I've searched this problem, and it supposed to be some ENDEIF missing, but it doesn't seems to be that in this case. here is the code from module_wrf_top.f90 from line 745 o 756

IF ( config_flags%check_TL .or. config_flags%check_AD ) THEN

   CALL allocate_grid ( )

   !$OMP PARALLEL DO    &
   !$OMP DEFAULT (SHARED) PRIVATE ( ij ) &
   DO ij = 1 , head_grid%num_tiles
      CALL copy_grid_to_s ( head_grid , &
                            head_grid%i_start(ij), head_grid%i_end(ij),                &
                            head_grid%j_start(ij), head_grid%j_end(ij)                 )
   ENDDO

does anyone know what could be the problem here?

davegill commented 5 years ago

@cross85 @kkeene44 @smileMchen Carlos, Two answers.

  1. Try getting rid of the trailing "&" on the second OpenMP directive. If this works for you, would you like to issue a pull request to fix the code?
  2. You have put together a nice question. Would you please repost this at http://forum.mmm.ucar.edu, probably under the "WRFDA SUPPORT" -> "Compiling / Installation" section.
ghost commented 5 years ago

@davegill It looks like there is also a missing "!$OMP END PARALLEL DO" after the do loop.

Carlos, are you attempting to compile WRFPLUS with OMP? To my knowledge, that capability is not supported at this time. @liujake ?

jjguerrette commented 5 years ago

Apologies, that comment was from me! I was logged into the wrong account.

davegill commented 5 years ago

@jjguerrette @cross85 JJ and Carlos, The impacted code is inside a large cpp WRFPLUS ifdef. I agree that there are likely two problems: the trailing "&" and the missing OpenMP directive to close the parallel section.

  1. If this portion of the WRF code is not expected to be able to be built by OpenMP, we should fix that in the configure step.

  2. Since there are several other OpenMP loops within this section of the code, we should verify if WRFPLUS and OpenMP is really a non-starter.

  3. Regardless of the WRFPLUS + OpenMP build decision, we should fix the Fortran/OpenMP syntax.

  4. Maybe we go with one of the release-v4.0.x branches (where "x" >= 3) for a PR the syntax error and a second PR to the develop branch for long-term WRFPLUS + OpenMP build restriction (if one exists).

liujake commented 5 years ago

I am not aware of WRFPlus working with OpenMP and fine to remove it or fix syntax error. Jake

cross85 commented 5 years ago

Thanks for your help. when you run configure you get this options:

cross@heisenberg:~/WRF/4.0.2_nc3/WRF_Plus$ ./configure wrfplus
checking for perl5... found /home/cross/Software/nc3/libs/perl5/bin/perl5 (perl5)
Will use NETCDF in dir: /home/cross/Software/nc3/libs/netcdf
HDF5 not set in environment. Will configure WRF for use without.
PHDF5 not set in environment. Will configure WRF for use without.
Will use 'time' to report timing information
$JASPERLIB or $JASPERINC not found in environment, configuring to build without grib2 I/O...
------------------------------------------------------------------------
Please select from among the following Linux x86_64 options:

  1. (serial)   2. (dmpar)   PGI (pgf90/gcc)
  3. (serial)   4. (dmpar)   PGI (pgf90/pgcc): SGI MPT
  5. (serial)   6. (dmpar)   PGI (pgf90/gcc): PGI accelerator
  7. (serial)   8. (dmpar)   INTEL (ifort/icc)
                             INTEL (ifort/icc): Xeon Phi (MIC architecture)
  9. (serial)  10. (dmpar)   INTEL (ifort/icc): Xeon (SNB with AVX mods)
 11. (serial)  12. (dmpar)   INTEL (ifort/icc): SGI MPT
 13. (serial)  14. (dmpar)   INTEL (ifort/icc): IBM POE
 15. (serial)  16. (dmpar)   PATHSCALE (pathf90/pathcc)
 17. (serial)  18. (dmpar)   GNU (gfortran/gcc)
 19. (serial)  20. (dmpar)   IBM (xlf90_r/cc_r)
 21. (serial)  22. (dmpar)   PGI (ftn/gcc): Cray XC CLE
 23. (serial)  24. (dmpar)   CRAY CCE (ftn $(NOOMP)/cc): Cray XE and XC
 25. (serial)  26. (dmpar)   INTEL (ftn/icc): Cray XC
 27. (serial)  28. (dmpar)   PGI (pgf90/pgcc)
 29. (serial)  30. (dmpar)   PGI (pgf90/gcc): -f90=pgf90
 31. (serial)  32. (dmpar)   PGI (pgf90/pgcc): -f90=pgf90
 33. (serial)  34. (dmpar)   INTEL (ifort/icc): HSW/BDW
 35. (serial)  36. (dmpar)   INTEL (ifort/icc): KNL MIC
 37. (serial)  38. (dmpar)   FUJITSU (frtpx/fccpx): FX10/FX100 SPARC64 IXfx/Xlfx

so you don't have an option to compile with OpenMP unless you change the configure.wrf

my settings for configure.wrf are these:

# Settings for    Linux x86_64 ppc64le i486 i586 i686, Xeon (SNB with AVX mods) ifort compiler with icc  (dmpar)
#

DESCRIPTION     =       INTEL ($SFC/$SCC): Xeon (SNB with AVX mods)
DMPARALLEL      =        1
OMPCPP          =       # -D_OPENMP
OMP             =       # -openmp -fpp -auto
OMPCC           =       # -openmp -fpp -auto
SFC             =       ifort
SCC             =       icc
CCOMP           =       icc
DM_FC           =       mpiifort
DM_CC           =       mpiicc -DMPI2_SUPPORT
FC              =       $(DM_FC)
CC              =       $(DM_CC) -DFSEEKO64_OK
LD              =       $(FC)
RWORDSIZE       =       8
PROMOTION       =       -r8        -real-size `expr 8 \* $(RWORDSIZE)` -i4
ARCH_LOCAL      =       -DNONSTANDARD_SYSTEM_FUNC -DCHUNK=64 -DXEON_OPTIMIZED_WSM5 -DOPTIMIZE_CFL_TEST  -DWRF_USE_CLM
OPTNOSIMD       =
OPTAVX          =       -xCORE-AVX512
CFLAGS_LOCAL    =       -w -O3 $(OPTAVX)
LDFLAGS_LOCAL   =       $(OPTAVX)
CPLUSPLUSLIB    =
ESMF_LDFLAG     =       $(CPLUSPLUSLIB)
FCOPTIM         =       -O3 $(OPTAVX)
FCREDUCEDOPT    =       $(FCOPTIM)
FCNOOPT         =       -O0 -fno-inline -no-ip
FCDEBUG         =       # -g $(FCNOOPT) -traceback # -fpe0 -check noarg_temp_created,bounds,format,output_conversion,pointers,uninit -ftrapuv -unroll0 -u
FORMAT_FIXED    =       -FI
FORMAT_FREE     =       -FR
FCSUFFIX        =
BYTESWAPIO      =       -convert big_endian
RECORDLENGTH    =       -assume byterecl
FCBASEOPTS_NO_G =       -w $(OMP) -auto -ftz -fno-alias -fp-model fast=1 -no-prec-div -no-prec-sqrt $(FORMAT_FREE) $(BYTESWAPIO) -auto -align array64byte #-vec-report6
FCBASEOPTS      =       $(FCBASEOPTS_NO_G) $(FCDEBUG)
MODULE_SRCH_FLAG =
TRADFLAG        =      -traditional-cpp
CPP             =      /lib/cpp -P -nostdinc
AR              =      ar
ARFLAGS         =      ru
M4              =      m4
RANLIB          =      ranlib
RLFLAGS         =
CC_TOOLS        =      gcc

When I checked the module_wrf_top.f90 I thought as the OMP options where commented wouldn't be a problem.

Now I'm going to compile again getting rid of the trailing "&" on the second OpenMP directive and adding "!$OMP END PARALLEL DO" after the do loop. So I'll tell you if I have luck compiling with these changes.

@davegill after compiling I'll write the post in the WRF forum and post the request to fix the code if it works.

cross85 commented 5 years ago

it worked! I changed the code of main/module_wrf_top.F to this:

IF ( config_flags%check_TL .or. config_flags%check_AD ) THEN

   ! Save the initial condition and boundary condition, x
   CALL allocate_grid ( )

   !$OMP PARALLEL DO    &
   !$OMP DEFAULT (SHARED) PRIVATE ( ij )
   !$OMP END PARALLEL DO
   DO ij = 1 , head_grid%num_tiles
      CALL copy_grid_to_s ( head_grid , &
                            head_grid%i_start(ij), head_grid%i_end(ij),                &
                            head_grid%j_start(ij), head_grid%j_end(ij)                 )
   ENDDO
davegill commented 5 years ago

@cross85 Carlos, Place the new !$OMP END PARALLEL DO after the ENDDO associated with the DO ij = 1 , head_grid%num_tiles. More like this:

   !$OMP PARALLEL DO    &
   !$OMP DEFAULT (SHARED) PRIVATE ( ij )
   DO ij = 1 , head_grid%num_tiles
      CALL copy_grid_to_s ( head_grid , &
                            head_grid%i_start(ij), head_grid%i_end(ij),                &
                            head_grid%j_start(ij), head_grid%j_end(ij)                 )
   ENDDO
   !$OMP END PARALLEL DO

Your version would compile and run, but it would not run in parallel for that loop.

cross85 commented 5 years ago

thanks @davegill, I'll do it that way.

jjguerrette commented 5 years ago

There would need to be additional testing to ensure that the TL/AD linear models are implemented properly for OMP. I do not know if those tests have ever been attempted. @mkavulich, were you working on testing WRFDA with OMP at some point? Do you have a sense of how much work would be involved for Carlos or the MMM DA team? @cross85, we may want to discuss your end goals in order to understand the need for this capability.

liujake commented 5 years ago

WRFDA does not support OMP and the same for WRFPlus. So I do not think we need to worry about OMP now. Jake

cross85 commented 5 years ago

@jjguerrette I wasn't trying to compile with OMP, I actually chose the option 10. (dmpar) INTEL (ifort/icc): Xeon (SNB with AVX mods) when I run the configure, and all OMP options were commmented, but anyways I had this error message when I was compiling WRFPLUS.

jjguerrette commented 5 years ago

Sorry about my confusion.

cross85 commented 5 years ago

@jjguerrette don't worry, it's very confusing because the bug is on the OMP part of the code even if it's disabled.

preritaagarwal commented 4 years ago

Hi,

I am trying to compile WRF-Chem code and I am doing it following this link [https://ruc.noaa.gov/wrf/wrf-chem/wrf_t ... _code.html].

However, it was not successful in the first attempt and I issued a clean -a command and tried to recompile. But now I see 'bash: compile: command not found' in the compile.log. I am not sure what went wrong.

Please could anyone help with this issue?

Thanks Prerita

weiwangncar commented 4 years ago

@preritaagarwal Please post the question on the Forum: https://forum.mmm.ucar.edu/phpBB3/. Thanks.