ufs-community / ufs-mrweather-app

UFS Medium-Range Weather Application
Other
23 stars 23 forks source link

Error building CCPP file radiation_aerosols.f with cime. #169

Closed uturuncoglu closed 3 years ago

uturuncoglu commented 4 years ago

@ligiabernardet @climbfuji We are getting following error when we try to build UFS MR Weather App under CIME. I am not sure it is a know issue or not.

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <END-OF-STATEMENT> ; <END-OF-FILE>

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <END-OF-STATEMENT> ; <END-OF-FILE>

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <LABEL> <END-OF-STATEMENT> ; <IDENTIFIER> TYPE MODULE ELEMENTAL IMPURE NON_RECURSIVE ...

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <LABEL> <END-OF-STATEMENT> ; <IDENTIFIER> TYPE MODULE ELEMENTAL IMPURE NON_RECURSIVE ...

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [=]

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <END-OF-STATEMENT> ; <END-OF-FILE>

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <END-OF-STATEMENT> ; <END-OF-FILE>

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <LABEL> <END-OF-STATEMENT> ; <IDENTIFIER> TYPE MODULE ELEMENTAL IMPURE NON_RECURSIVE ...

/glade/u/home/turuncu/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5082: Syntax error, found '==' when expecting one of: <LABEL> <END-OF-STATEMENT> ; <IDENTIFIER> TYPE MODULE ELEMENTAL IMPURE NON_RECURSIVE ...

In this case, i am using release/public-v1 branch for the model and last commit is

commit 8cb21f8a40c8b262bc1b3c0c32f321fdb3244a5c
Author: JulieSchramm <schramm@ucar.edu>
Date:   Wed Jul 15 15:58:23 2020 -0600

If you could also create a case on Cheyenne using UFS MR Weather Model (or I could do it if you provide me the instructions) that could help to compare the possible differences in the build. BTW, I am using intel compiler and MPT combination.

hertneky commented 4 years ago

@uturuncoglu that works now - thanks

hertneky commented 4 years ago

@uturuncoglu Testing flat file customization complete and working as visualized/intended for updated v1.1 on Cheyenne. See #174 for details.

uturuncoglu commented 4 years ago

@ligiabernardet @climbfuji I am testing model on Stampede and trying to create a new baseline with Intel compiler. The build is giving similar error also in that platform. It is not reproducible and some cases build without any problem.

Finished MODEL_BUILD for test SMS_Lh3_D.C96.GFSv15p2.stampede2-skx_intel in 63.245402 seconds (FAIL). [COMPLETED 1 of 6]
    Case dir: /scratch/01118/tg803972/SMS_Lh3_D.C96.GFSv15p2.stampede2-skx_intel.G.20200904_154442_da4ltg
    Errors were:
        Building test for SMS in directory /scratch/01118/tg803972/SMS_Lh3_D.C96.GFSv15p2.stampede2-skx_intel.G.20200904_154442_da4ltg
        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [l]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [s]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [l]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [s]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [l]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [s]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [l]

        /home1/01118/tg803972/UFS/ufs-mrweather-app/src/model/FV3/ccpp/physics/physics/radiation_aerosols.f(5715): error #5149: Illegal character in statement label field  [s]

        ERROR: BUILD FAIL: ufsatm.buildlib failed, cat /scratch/01118/tg803972/SMS_Lh3_D.C96.GFSv15p2.stampede2-skx_intel.G.20200904_154442_da4ltg/bld/atm.bldlog.200904-154514
uturuncoglu commented 4 years ago

@ligiabernardet @climbfuji I am getting /work/noaa/nems/tufuk/scratch/SMS_Lh3.C96.GFSv15p2.orion_intel.20200908_163017_0550g2/bld/atm/obj/FV3/ccpp/physics/physics/radiation_aerosols.f(2197): error #5082: Syntax error, found '-' when expecting one of: <LABEL> <END-OF-STATEMENT> ; <IDENTIFIER> TYPE MODULE ELEMENTAL IMPURE NON_RECURSIVE ... type errors also on Orion.

climbfuji commented 4 years ago

@uturuncoglu how exactly are you getting the code onto orion (and cheyenne)?

uturuncoglu commented 4 years ago

@climbfuji i am just using manage externals. Most of the test cases are failing with build error.

uturuncoglu commented 4 years ago

@climbfuji For example following test failed with build error, /work/noaa/nems/tufuk/scratch/SMS_Lh3_D.C96.GFSv16beta.orion_intel.20200908_163017_0550g2 and the log is in here /work/noaa/nems/tufuk/scratch/SMS_Lh3_D.C96.GFSv16beta.orion_intel.20200908_163017_0550g2/bld/atm.bldlog.200908-163714.

uturuncoglu commented 4 years ago

@climbfuji FYI, only 6/19 tests are PASSED and others are failed with build error on Orion.

uturuncoglu commented 4 years ago

@climbfuji I compared application versions 1.0 and the 1.1 to find any difference in the compile options for the radiation_aerosols.f file and both of them are compiled exactly same options. Also, I run full test suite with 1.0 also and I did not see any compile time error. The CHGRES is also failing for some cases in 1.0, which was working in the 1.0 release time (https://github.com/ufs-community/ufs-mrweather-app/issues/179). This indicates some changes in the system level. At this point the only difference seems model versions. I'll also compare the buildlib between 1.0 and 1.1 to find any particular reason that could cause the build error.

climbfuji commented 4 years ago

@climbfuji I compared application versions 1.0 and the 1.1 to find any difference in the compile options for the radiation_aerosols.f file and both of them are compiled exactly same options. Also, I run full test suite with 1.0 also and I did not see any compile time error. The CHGRES is also failing for some cases in 1.0, which was working in the 1.0 release time (#179). This indicates some changes in the system level. At this point the only difference seems model versions. I'll also compare the buildlib between 1.0 and 1.1 to find any particular reason that could cause the build error.

I do get the build error as well when I use the CIME regression tests. The changes in the model code itself between 1.0 and 1.1 are tiny and have nothing to do with actual source code (Python 3 support for the CCPP code generator, changes in namelist templates). This smells a lot like some changes in CIME causing the compilation to fail.

uturuncoglu commented 4 years ago

@climbfuji Okay that is fine. I am looking into CIME to find anything. I'll try to test the build by setting GMAKE_J=1 with xmlchange. The default value is 8. To see, I could reproduce the issue also with GMAKE_J=1.

ligiabernardet commented 3 years ago

This problem stopped happening after @uturuncoglu reverted to the MRW 1.0 buildlib.