search

ufs-community / UFS_UTILS

Utilities for the NCEP models.
Other
21 stars 104 forks source link

CHGRES_CUBE - range check SST #59

Closed GeorgeGayno-NOAA closed 4 years ago

GeorgeGayno-NOAA commented 4 years ago

During testing for the public release, the model failed (in debug mode) when using coldstart files prepared from GFS GRIB2 files. The failure - in the nsst_module - indicates a too cold SST. Investigate.

See this issue for more details: https://github.com/ufs-community/ufs-mrweather-app/issues/87

GeorgeGayno-NOAA commented 4 years ago

Too cold (< 271.2K) SST values were noted when using v14, v15.1 and v15.2 GFS GRIB2 data. Also, a sample file from one of Russ' v16 parallels also had cold values. These cold values are the result of bi-linear interpolation from the gaussian master files to the lat/lon "pgb" files. As a result, coastal points (even those identified as water by the mask) can be a mixture of land and non-land. The 0.5-degree data file had, on average, 300 cold points. Points with SSTs that were too warm (> 310K) were much fewer (one or two points at most).

GeorgeGayno-NOAA commented 4 years ago

Tried to reproduce the model crash on Hera and Dell (https://github.com/ufs-community/ufs-mrweather-app/issues/87) with no luck. I used the GFS.v16.0.0 tag of the model and compiled it in debug mode. I ran it for a global C96 canned case. Even if I intentionally add very cold points (200 K) to the initial conditions, it still runs. Not sure why that is. Nevertheless, this problem should be fixed. Will ask Xu Li for assistance.

GeorgeGayno-NOAA commented 4 years ago

After consulting with Jun, she recommended I try to reproduce the error with the public release branch. After some trial and error, I was able to do so. What I did:

The model broke before the write of the 00 hour history file:

131: Image PC Routine Line Source 131: fv3.exe 0000000003C60748 nst_module_mp_coo 894 module_nst_model.f90 131: fv3.exe 00000000036CC978 sfc_nst_mp_sfc_ns 392 sfc_nst.f

That is the same line of code as the Cheyenne test: https://github.com/ufs-community/ufs-mrweather-app/issues/87

GeorgeGayno-NOAA commented 4 years ago

The previous test was then repeated using the feature/chgres_sst branch at e1fdb41. The branch contains Xu Li's recommended range-checking of SST. The test ran successfully for 48 hours. Xu Li and I examined the output and it looked correct.

Will merge to 'develop'.

GeorgeGayno-NOAA commented 4 years ago

Some final checks before merging.

The branch at b3c75cf was compiled (all programs) on Hera, Jet, Cray and Dell with no problems.

The chgres_cube regression test suite was run on Hera, Jet, Cray and Dell. As expected the GRIB2 test failed. Differences were noted in tiles 3 and 6.

Tile 3 Variable Group Count Sum AbsSum Min Max Range Mean StdDev tsea / 307 69.8411 69.8411 7.05186e-06 3.71142 3.71141 0.227495 0.501393 tg3 / 307 69.8411 69.8411 7.05186e-06 3.71142 3.71141 0.227495 0.501393 tisfc / 307 69.8411 69.8411 7.05186e-06 3.71142 3.71141 0.227495 0.501393 stc / 1228 279.364 279.364 7.05186e-06 3.71142 3.71141 0.227495 0.50078

Tile 6 Variable Group Count Sum AbsSum Min Max Range Mean StdDev tsea / 328 10.322 10.322 3.22052e-09 0.344322 0.344322 0.0314695 0.0441827 tg3 / 328 10.322 10.322 3.22052e-09 0.344322 0.344322 0.0314695 0.0441827 tisfc / 328 10.322 10.322 3.22052e-09 0.344322 0.344322 0.0314695 0.0441827 stc / 1312 41.2879 41.2879 3.22052e-09 0.344322 0.344322 0.0314695 0.0441321

These tiles cover the north and south poles respectively. Since the GRIB2 data can contain hundreds of too cold SSTs, differences in these tiles (variable 'tsea') was expected. Note, at open water variables 'tg3', 'tisfc' and 'stc' are not used and contain flag values (which are SST).

GeorgeGayno-NOAA commented 4 years ago

Fix was approved and merged to develop at 2d7f796. Will remove my feature/chgres_sst branch and close issue.