ufs-community / UFS_UTILS

Utilities for the NCEP models.
Other
21 stars 104 forks source link

CHGRES_CUBE - sea ice range check #32

Closed GeorgeGayno-NOAA closed 4 years ago

GeorgeGayno-NOAA commented 4 years ago

Dusan used chgres_cube to initialize a C96 global run using FV3GFS gaussian nemsio data as input (2019111600 cycle). The model broke during on the first time step during a computation of surface stress. In his words:

I recompiled the model with debug flags and found that model crashes in 
the very first physics step, in moninedmf.f (subroutine hedmf_run) at this line:

 411       do i=1,im
 412          ustar(i) = sqrt(stress(i))
 413       enddo

At one point stress(i) is negative number. It is computed in GFS_surface_composites.F90 here:

472    stress(i) = cice(i)*stress_ice(i) + (one-cice(i))*stress_ocn(i)    !<--- cice(i) must not be > 1.0

At that point, the sea ice fraction was slightly more than one (1.0000004). When he reset that point to exactly one, the model ran with no issues.

Determine why this is happening in chgres and add checks to prevent it.

GeorgeGayno-NOAA commented 4 years ago

Dusan tried the head of the branch (289d36a) and it worked. Will run the regression tests next.

GeorgeGayno-NOAA commented 4 years ago

The regression tests were run on Cray, Dell, Hera and Jet. Some tests passed and some failed:

<<< C96 FV3 RESTART TEST FAILED. >>> <<< C192 FV3 HISTORY TEST PASSED. >>> <<< C96 FV3 GAUSSIAN NEMSIO TEST FAILED. >>> <<< C96 GFS SIGIO TEST FAILED. >>> <<< C96 GFS GAUSSIAN NEMSIO TEST FAILED. >>> <<< C96 REGIONAL TEST PASSED. >>>

All failures were the result of round-off differences in fractional ice. For example, output from the nccmp utility for one tile showed:

Variable Group Count Sum AbsSum Min Max Range Mean StdDev fice / 1 -2.22045e-16 2.22045e-16 -2.22045e-16 -2.22045e-16 0 -2.22045e-16 0

This is an expected result. Will merge to 'develop'.

GeorgeGayno-NOAA commented 4 years ago

Work completed. Will remove _feature/chgres_icecheck branch and close ticket.