TRENDY-SP: ctsm51d142_f19_CRUjra_[1850, hist]

wwieder commented 9 months ago

Description: Assess land model results from and SP cased forced with TRENDY (CRU-JRA) datm. I'm inclined to do this with a HIST compset that's been spun up for a bit first? I also think we could do this with with a 1 or 2 degree FV grid (f09... or f19...)

Same as #18 but with CRUjra. For guidance, looked at Daniel Kennedy's TRENDY case /glade/u/home/djk2120/ctsm_trendy_2023/sims/TRENDY2023_f09_S3

Spin-up: ./create_newcase --compset I1850Clm51Sp --res f19_f19 --case ~/cases_LMWG_dev/ctsm51d142_f19_CRUjra_1850 --run-unsupported Hist: ./create_newcase --compset IHistClm51Sp --res f19_f19 --case ~/cases_LMWG_dev/ctsm51d142_f19_CRUjra_hist --run-unsupported

Case directory: Spin-up: /glade/u/home/slevis/cases_LMWG_dev/ctsm51d142_f19_CRUjra_1850 Hist part 1: /glade/u/home/slevis/cases_LMWG_dev/ctsm51d142_f19_CRUjra_hist.1850-1919 Hist: /glade/u/home/slevis/cases_LMWG_dev/ctsm51d142_f19_CRUjra_hist hist directory is a copy of "hist part 1" with updates to user_nl_clm and env_run.

Sandbox: /glade/work/slevis/git/latest_master git describe: ctsm5.1.dev142

usernl changes:

user_nl_clm

spin-up: NONE hist part 1:

finidat = '/glade/scratch/slevis/archive/ctsm51d142_f19_CRUjra_1850/rest/0021-01-01-00000/ctsm51d142_f19_CRUjra_1850.clm2.r.0021-01-01-00000.nc'
use_init_interp = .true.

hist: back to NONE; change env_run to hybrid starting in 1920 and cycle datm to 2022

user_nl_datm_streams (note "datafiles" strings have been shortened below)

CLMCRUNCEPv7.Solar:tintalgo = coszen
CLMCRUNCEPv7.Solar:mapalgo = bilinear
CLMCRUNCEPv7.Solar:taxmode = cycle
CLMCRUNCEPv7.Solar:datavars = FSDS_DIRECT Faxa_swdndr,FSDS_DIFFUSE Faxa_swdndf
CLMCRUNCEPv7.Solar:meshfile = /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/lnd_mesh_cdf5.nc
CLMCRUNCEPv7.Solar:datafiles = /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.Solr.1901_cdf5.nc,...,/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.Solr.2022_cdf5.nc
CLMCRUNCEPv7.Precip:tintalgo = nearest
CLMCRUNCEPv7.Precip:mapalgo = bilinear
CLMCRUNCEPv7.Precip:taxmode = cycle
CLMCRUNCEPv7.Precip:datavars = PRECTmms Faxa_precn
CLMCRUNCEPv7.Precip:meshfile = /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/lnd_mesh_cdf5.nc
CLMCRUNCEPv7.Precip:datafiles = /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.Prec.1901_cdf5.nc,...,/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.Prec.2022_cdf5.nc
CLMCRUNCEPv7.TPQW:tintalgo = linear
CLMCRUNCEPv7.TPQW:mapalgo = bilinear
CLMCRUNCEPv7.TPQW:taxmode = cycle
CLMCRUNCEPv7.TPQW:datavars = QBOT Sa_shum,PSRF Sa_pbot,TBOT Sa_tbot,WIND Sa_wind
CLMCRUNCEPv7.TPQW:meshfile = /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/lnd_mesh_cdf5.nc
CLMCRUNCEPv7.TPQW:datafiles = /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.TPQWL.1901_cdf5.nc,...,/glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.TPQWL.2022_cdf5.nc

SourceMods: NONE

Diagnostics: https://webext.cgd.ucar.edu/I20TR/ctsm51d142_f19_CRUjra_hist/lnd/ctsm51d142_f19_CRUjra_hist-ctsm51d142_f19_GSWP3V1_hist/setsIndex.html

Output: Output (if still available): /glade/campaign/cgd/tss/people/slevis/i_cases/cases_LMWG_dev/ctsm51d142_f19_CRUjra_1850/ /glade/campaign/cgd/tss/people/slevis/i_cases/cases_LMWG_dev/ctsm51d142_f19_CRUjra_hist/

Contacts: @slevis-lmwg

Extra details: Spin-up phase: 20 years with DATM_YR_START 1901 and DATM_YR_END 1920. Hist part 1: 1850-1919 with DATM_YR_START 1901, DATM_YR_END 1920, and DATM_YR_ALIGN 1901. Hist: 1920-2022 with DATM_YR_START 1901, DATM_YR_END 2022, and DATM_YR_ALIGN 1901.

slevis-lmwg commented 8 months ago

Made mesh file for this datm dataset because it didn't exist, yet, as far as I could tell.

slevis-lmwg commented 8 months ago

The run fails for now. A few issues that I have resolved:

Converted the datm files from netcdf4 to cdf5
In user_nl_datmstreams, I added prefixes Faxa and Sa_ to the variable names (error messages were unhelpful)

slevis-lmwg commented 8 months ago

Progress! The run fails with a new error beyond where it stopped before.

DATM output stops here:

 atm : model date        10101           0
 atm : model date        10101        1800
 atm : model date        10101        3600
 atm : model date        10101        5400
 atm : model date        10101        7200
 atm : model date        10101        9000
(shr_strdata_readstrm) reading file ub: /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.Prec.1901_cdf5.nc       2
(shr_strdata_readstrm) reading file ub: /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.TPQWL.1901_cdf5.nc       2
 atm : model date        10101       10800
 atm : model date        10101       12600
 atm : model date        10101       14400
 atm : model date        10101       16200
 atm : model date        10101       18000
 atm : model date        10101       19800
(shr_strdata_readstrm) reading file ub: /glade/p/cgd/tss/CTSM_datm_forcing_data/atm_forcing.datm7.CRUJRA.0.5d.v1.c190604/TRENDY_cdf5/clmforc.TRENDY.c2023_0.5x0.5.Solr.1901_cdf5.nc       3
 atm : model date        10101       21600

I'm guessing the above relates to the lnd-atm coupling frequency.

The error appears in the lnd and cesm log files:

# of NaNs =            1
Which are NaNs =  F T F F
NaN found in field Sl_t at gridcell index            2

Traceback:

lnd_import_export         165  lnd_import_export_utils.F90
lnd_import_export        1170  lnd_import_export.F90
lnd_import_export         761  lnd_import_export.F90
lnd_comp_nuopc_mp         912  lnd_comp_nuopc.F90

My current best guess is that this relates to TBOT in CRUJRA having zeros over ocean. In GSWP3 and CRUv7 there's data over ocean. I'm considering two options and would like feedback @olyson @wwieder:

Set the zeros to 1e36. If you think that this will work, I likely need to do it for most or all variables.
Fill the zeros with data from elsewhere. From what I can tell, we filled CRUv7 with Qian, so this would be along the same lines.

olyson commented 8 months ago

I'm wondering what the datm domain file (or I guess mesh in this case) has for these zero value grid cells. Is there a mask in the mesh file that should take care of this (tell the datm where the valid values are?).

slevis-lmwg commented 8 months ago

Very good point: the mesh file's mask contains ones everywhere. I should run it through the mesh_mask_modifier. (This occurred to me at an earlier stage, and then I forgot it.) Update: Didn't need mesh_mask_modifier, just needed to start from a landmask.nc with the correct mask in it.

Another question: The files contain FLDS, but Daniel did not seem to use it. I am now thinking that I should use it. Feedback on that?

olyson commented 8 months ago

My recollection is that we had decided not to use FLDS in the CRUv7 because of questionable values at high latitudes. I'm not sure where the downward longwave comes from in CRUJRA.

slevis-lmwg commented 8 months ago

It's running now! I kept FLDS out of it, since Daniel did, too.

slevis-lmwg commented 8 months ago

Diagnostics posted above

wwieder commented 8 months ago

Thanks for getting this done, Sam! It looks like seasonal maps (set. 2) didn't process or upload correctly (only DJF results shown). Maybe we can check after the machines are back up.

wwieder commented 8 months ago

BTW, antarctic runoff looks crazy, but I'm not sure why?

slevis-lmwg commented 8 months ago

BTW, antarctic runoff looks crazy, but I'm not sure why?

Temperature and Precip seem very different and probably wrong over Antarctica. FSDS in set2 also looks strange. Given that CRUjra is for use with TRENDY, I wonder if the data in Antarctica are an afterthought.

slevis-lmwg commented 8 months ago

Diagnostics updated.

wwieder commented 8 months ago

Should we look at ILAMB comparison of GSWP3 & CRU-JRA results, hopefully adding ERA5 when it's available, or just wait for ERA5?

wwieder commented 7 months ago

@olyson volunteered to run ilamb on #18, #19, & #20. We'll wait on ERA5, as these datm inputs are still forthcoming.

olyson commented 7 months ago

ILAMB results for GSWP3V1, CRUNCEPV7, and CRUJRA here:

https://webext.cgd.ucar.edu/I20TR/ctsm51d142_f19_GSWP3V1_hist/lnd/_build_ILAMB2_7_ForcingDatasets/

NCAR / LMWG_dev

TRENDY-SP: ctsm51d142_f19_CRUjra_[1850, hist] #20