search

ESMCI / cmeps-cime

This is a "fork" of the cime repository that has the development version of the nuopc CMEPS driver and mediator
Other
0 stars 0 forks source link

use single branch for both CESM and FV3 applications #88

Closed uturuncoglu closed 5 years ago

uturuncoglu commented 5 years ago

This changes aims to use single branch for both CESM and FV3 applications.

Note about component hashes: UFSCOMP: Need to modify Externals.cfg for CICE to point single branch for both CESM and FV3 applications (branch = mvertens/nuopc_cap) CIME: e57f162

Test suite: There is no specific test suite for FV3 but following commands can be used to check build and run

./create_newcase --compset UFS_S2S --res C384_t025 --case ufs.s2s.c384_t025.jan --driver nuopc --run-unsupported cd ufs.s2s.c384_t025.jan/ ./case.setup ./xmlchange DOUT_S=FALSE ./xmlchange STOP_N=1 ./xmlchange RUN_REFDATE=2012-01-01 ./xmlchange RUN_STARTDATE=2012-01-01 ./xmlchange JOB_WALLCLOCK_TIME=01:00:00 qcmd -- ./case.build ./case.submit

To test CESM application:

The following command can be used to test app_cesm-cmeps. Tester will see FAIL in all NLCOMP phases due to the change in the namelist files such as ice namelist and change in the PET layouts of ice component because the ice branches are merged together to have single ice branch for FV3 and CESM applications. It would be better to create new baseline.

qcmd -l walltime=4:00:00 -- ./create_test --xml-testlist ../src/drivers/nuopc/cime_config/testdefs/testlist_drv.xml --xml-machine cheyenne --xml-category nuopc --compare apr23intel19 --baseline-root /glade/p/cesmdata/cseg/nuopc_baselines

Test baseline: N/A Test namelist changes: N/A Test status: N/A

Fixes [CIME Github issue #]

User interface changes?:

Update gh-pages html (Y/N)?:

Code review:

rsdunlapiv commented 5 years ago

@uturuncoglu @jedwards4b Travis-CI failed:

FAIL: test_pylint_src_drivers_nuopc_cime_config_buildnml (__main__.B_CheckCode)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./scripts_regression_tests.py", line 3204, in test
    self.assertTrue(result == "", msg=result)
AssertionError: False is not true : ************* Module buildnml
W: 76, 0: Found indentation with tabs instead of spaces (mixed-indentation)
W: 78, 0: Found indentation with tabs instead of spaces (mixed-indentation)

Indentation error (tabs instead of spaces)...

I will proceed with tests since this only affects Python. Please push change when ready.

rsdunlapiv commented 5 years ago

@uturuncoglu

Creating the FV3 compset failed:

dunlap@cheyenne5:~/UFSCOMP.pr88/cime/scripts> ./create_newcase --compset UFS_S2S --res C384_t025 --case ufs.s2s.c384_t025.jan --driver nuopc --run-unsupported
Did not find an alias or longname compset match for UFS_S2S
rsdunlapiv commented 5 years ago

Ah - this must be because FV3 is not in the Externals.cfg. I will add and try again.

rsdunlapiv commented 5 years ago

New error on case.build for FV3 case:

dunlap@cheyenne5:~/UFSCOMP.pr88/cime/scripts/ufs.s2s.c384_t025.jan> qcmd -- ./case.build
Submitting command to PBS using account P93300606:
    ./case.build

Waiting for job 5373437.chadmin1 to start ... 

Building case in directory /gpfs/u/home/dunlap/UFSCOMP.pr88/cime/scripts/ufs.s2s.c384_t025.jan
sharedlib_only is False
model_only is False
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Generating component namelists as part of build
Creating component namelists
  2019-04-26 13:32:53 atm
   Calling /gpfs/u/home/dunlap/UFSCOMP.pr88/components/fv3//cime_config/buildnml
    Creating INPUT and RESTART directory
    Copying input files
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/*configure /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/global_o3prdlos.f77 /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/co2historicaldata_201*.txt /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/*grb /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/model_configure.cmeps /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/model_configure
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/aerosol.dat /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/INPUT/* /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/INPUT
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/input.nml.cmeps /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/input.nml
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/sfc_emissivity_idx.txt /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/global_h2oprdlos.f77 /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/co2historicaldata_glob.txt /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/*_table /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/benchmark-inputs/2012010100/gfs/fcst/solarconstant_noaa_an.txt /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/
cp /glade/work/turuncu/FV3GFS/addon/C384/diag_table.cmeps /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/diag_table
cp /glade/work/turuncu/FV3GFS/addon/C384/grid_spec.nc /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run/INPUT/grid_spec.nc
    Linking input directories
    Linking input static input files
  2019-04-26 13:33:00 lnd
   Calling /gpfs/u/home/dunlap/UFSCOMP.pr88/cime/src/components/stub_comps/slnd/cime_config/buildnml
  2019-04-26 13:33:00 ice
   Calling /gpfs/u/home/dunlap/UFSCOMP.pr88/components/cice//cime_config/buildnml
     ...buildnml calling cice buildcpp to set build time options
  2019-04-26 13:33:00 ocn
   Calling /gpfs/u/home/dunlap/UFSCOMP.pr88/components/mom//cime_config/buildnml
ERROR: Couldn't find input files for a%C384_l%null_oi%tx0.25v1_r%null_g%null_w%null_z%null_m%tx0.25v1

The issue might have to do with the MOM interface branches that need to be merged, i.e.:

https://github.com/ESCOMP/MOM_interface/tree/dev/nuopc_cmeps_fv3 and https://github.com/ESCOMP/MOM_interface/tree/nuopc_branch

uturuncoglu commented 5 years ago

Yes. I merged following branches: https://github.com/ESCOMP/MOM_interface.git dev/nuopc_cmeps_fv3 branch is merged to nuopc_branch Could you test it again? You might need to update MOM_interface

rsdunlapiv commented 5 years ago

@uturuncoglu The FV3 runs has some ERRORs in the PET logs:

See /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run

20190426 143857.656 ERROR            PET0000 ESMF_StateAPI.cppF90:3103 ESMF_StateGet         Not found  -  no ESMF_Field found named: cpl_scalars
20190426 143857.657 ERROR            PET0000 shr_nuopc_methods_mod.F90:3205         Not found  - Passing error in return code
20190426 143857.657 ERROR            PET0000 ESMF_StateAPI.cppF90:3103 ESMF_StateGet         Not found  -  no ESMF_Field found named: cpl_scalars
20190426 143857.657 ERROR            PET0000 shr_nuopc_methods_mod.F90:3205         Not found  - Passing error in return code

I suspect this is due to the atmosphere not providing cpl_scalars? It may be that we just need to remove the error message from the mediator, because the run did finish. However, it will be confusing for users if they see the errors.

mvertens commented 5 years ago

Yes - we should remove these.

On Fri, Apr 26, 2019 at 3:05 PM Rocky Dunlap notifications@github.com wrote:

@uturuncoglu https://github.com/uturuncoglu The FV3 runs has some ERRORs in the PET logs:

See /glade/scratch/dunlap/ufs.s2s.c384_t025.jan/run

20190426 143857.656 ERROR PET0000 ESMF_StateAPI.cppF90:3103 ESMF_StateGet Not found - no ESMF_Field found named: cpl_scalars 20190426 143857.657 ERROR PET0000 shr_nuopc_methods_mod.F90:3205 Not found - Passing error in return code 20190426 143857.657 ERROR PET0000 ESMF_StateAPI.cppF90:3103 ESMF_StateGet Not found - no ESMF_Field found named: cpl_scalars 20190426 143857.657 ERROR PET0000 shr_nuopc_methods_mod.F90:3205 Not found - Passing error in return code

I suspect this is due to the atmosphere not providing cpl_scalars? It may be that we just need to remove the error message from the mediator, because the run did finish. However, it will be confusing for users if they see the errors.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ESMCI/cmeps-cime/pull/88#issuecomment-487201150, or mute the thread https://github.com/notifications/unsubscribe-auth/AB4XCEZDZ4I5477R7ZHU2BDPSNVDBANCNFSM4HIZURAQ .

rsdunlapiv commented 5 years ago 
dunlap@cheyenne5:~/UFSCOMP.pr88/cime/scripts> qcmd -l walltime=4:00:00 -- ./create_test --xml-testlist ../src/drivers/nuopc/cime_config/testdefs/testlist_drv.xml --xml-machine cheyenne --xml-category nuopc --compare apr23intel19 --generate apr26intel19 --baseline-root /glade/p/cesmdata/cseg/nuopc_baselines
rsdunlapiv commented 5 years ago

@uturuncoglu Failures in the test suite:

/glade/scratch/dunlap/ERS_Vnuopc_Ln5.f19_g17.F2000Nuopc.cheyenne_intel.cam-nuopc_cap.GC.20190426_174721_p8r2u2/run

20190426 183711.143 ERROR            PET70     (ice_comp_nuopc):(ModelAdvance)  CICE clock not in sync with ESMF model clock
20190426 183711.143 ERROR            PET70 ICE:src/addon/NUOPC/src/NUOPC_ModelBase.F90:907     Failure  - Passing error in return code
20190426 183711.143 ERROR            PET70 ESMF_Comp.F90:1239 ESMF_CompExecute     Wrong argument specified  - Internal subroutine call returned Error
20190426 183711.143 ERROR            PET70 ESMF_GridComp.F90:1903 ESMF_GridCompRun     Wrong argument specified  - Internal subroutine call returned Error
20190426 183711.143 ERROR            PET70 ESM0001:src/addon/NUOPC/src/NUOPC_Driver.F90:2878     Wrong argument specified  - Failed calling phase RunPhase1 Run for modelComp 3
20190426 183711.143 ERROR            PET70 ESMF_Comp.F90:1239 ESMF_CompExecute     Wrong argument specified  - Internal subroutine call returned Error
20190426 183711.143 ERROR            PET70 ESMF_GridComp.F90:1903 ESMF_GridCompRun     Wrong argument specified  - Internal subroutine call returned Error
20190426 183711.143 ERROR            PET70 ensemble:src/addon/NUOPC/src/NUOPC_Driver.F90:2878     Wrong argument specified  - Failed calling phase RunPhase1 Run for modelComp 1
20190426 183711.143 ERROR            PET70 ESMF_Comp.F90:1239 ESMF_CompExecute     Wrong argument specified  - Internal subroutine call returned Error
20190426 183711.143 ERROR            PET70 ESMF_GridComp.F90:1903 ESMF_GridCompRun     Wrong argument specified  - Internal subroutine call returned Error
2

This case timed out: /glade/scratch/dunlap/ERS_Vnuopc_Ln9.f19_g17.X.cheyenne_intel.GC.20190426_174721_p8r2u2 This time for this case should be increased in: cime/src/drivers/nuopc/cime_config/testdefs/testlist_drv.xml

I also had GENERATE failures because I do not have permissions to the baselines directory. I sent an email to be added to the cseg group.

uturuncoglu commented 5 years ago

The runtype namelist option is removed from cime_config/namelist_definition_cice.xml because the default value (initial) was causing the clock synchronization problem. This must be set it by mediator as it is before. So, runtype namelist option is commented again.

rsdunlapiv commented 5 years ago

@uturuncoglu @mvertens The DTEST shows baseline differences in cice and cpl history files: /glade/scratch/dunlap/SMS_Vnuopc_Ld5.T62_g37.DTEST.cheyenne_intel.cice-nuopc_cap.GC.20190429_145109_sd3zu2

Is this expected due to CICE namelist changes?

rsdunlapiv commented 5 years ago

Generated new baselines: /glade/p/cesmdata/cseg/nuopc_baselines/may1intel19

All failures are as expected:

dunlap@cheyenne5:/glade/scratch/dunlap> ./cs.status.20190501_122440_o8kp22 | grep "FAIL "
    FAIL ERR_Vnuopc_Ld5.f19_g16.BMOM.cheyenne_intel.allactive-nuopc_cap_io NLCOMP
    FAIL ERR_Vnuopc_Ld5.f19_g16.BMOM.cheyenne_intel.allactive-nuopc_cap_io COMPARE_base_rest
    FAIL ERS_Vnuopc_Ld5.T62_g16.CMOM.cheyenne_intel NLCOMP
    FAIL ERS_Vnuopc_Ld5.T62_g16.CMOM.cheyenne_intel COMPARE_base_rest
    FAIL ERS_Vnuopc_Ld5.T62_g16.GMOM.cheyenne_intel NLCOMP
    FAIL ERS_Vnuopc_Ld5.T62_g16.GMOM.cheyenne_intel COMPARE_base_rest
    FAIL ERS_Vnuopc_Ld5.T62_g37.DTEST.cheyenne_intel.cice-nuopc_cap NLCOMP
    FAIL ERS_Vnuopc_Ld5.T62_g37.DTEST.cheyenne_intel.cice-nuopc_cap BASELINE apr23intel19: DIFF
    FAIL ERS_Vnuopc_Ln5.f19_g17.F2000Nuopc.cheyenne_intel.cam-nuopc_cap NLCOMP
    FAIL ERS_Vnuopc_Ln5.f45_f45_mg37.I2000Clm50SpNuopc.cheyenne_intel.clm-nuopc_cap NLCOMP
    FAIL ERS_Vnuopc_Ln5.f45_f45_mg37.I2000Clm50SpNuopc.cheyenne_intel.clm-nuopc_cap BASELINE apr23intel19: ERROR BFAIL some baseline files were missing
    FAIL ERS_Vnuopc_Ln5.ne16_ne16_mg17.QPC4.cheyenne_intel.cam-nuopc_cap NLCOMP
    FAIL ERS_Vnuopc_Ln9.f19_g17.X.cheyenne_intel NLCOMP
    FAIL ERS_Vnuopc_Ln9.f19_g17.X.cheyenne_intel BASELINE apr23intel19: ERROR BFAIL some baseline files were missing
    FAIL ERS_Vnuopc_Ln9_N3.f19_g17_rx1.A.cheyenne_intel NLCOMP
    FAIL ERS_Vnuopc_Ln9_N3.f19_g17_rx1.A.cheyenne_intel TPUTCOMP Error: Computation time increase > 25 pct from baseline
    FAIL SMS_Vnuopc.f19_g17.X.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ld1_N3.f19_g17_rx1.A.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ld1_N3.f19_g17_rx1.A.cheyenne_intel TPUTCOMP Error: Computation time increase > 25 pct from baseline
    FAIL SMS_Vnuopc_Ld2.ww3a.ADWAV.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ld3.f09_f09_mg17.A1850DLND.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ld5.f19_g16.BMOM.cheyenne_intel.allactive-nuopc_cap_io NLCOMP
    FAIL SMS_Vnuopc_Ld5.T62_g16.CMOM.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ld5.T62_g16.GMOM.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ld5.T62_g37.DTEST.cheyenne_intel.cice-nuopc_cap NLCOMP
    FAIL SMS_Vnuopc_Ld5.T62_g37.DTEST.cheyenne_intel.cice-nuopc_cap BASELINE apr23intel19: DIFF
    FAIL SMS_Vnuopc_Ln11_D.f19_g17_rx1.A.cheyenne_intel NLCOMP
    FAIL SMS_Vnuopc_Ln5.f19_f19_mg17.I2000Clm50SpGs.cheyenne_intel.clm-nuopc_cap NLCOMP
    FAIL SMS_Vnuopc_Ln5.f19_g17.F2000Nuopc.cheyenne_intel.cam-nuopc_cap NLCOMP
    FAIL SMS_Vnuopc_Ln5.f45_f45_mg37.I2000Clm50SpNuopc.cheyenne_intel.clm-nuopc_cap NLCOMP
    FAIL SMS_Vnuopc_Ln5.ne16_ne16_mg17.QPC4.cheyenne_intel.cam-nuopc_cap NLCOMP