f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001

[cecilehannay]

Description: New control with cam6_3_107 + clm 5.1 physics as discussed in: #249

compset:

HIST_CAM%DEV%LT%GHGMAM4_CLM51%SP_CICE%PRES_DOCN%DOM_MOSART_SGLC_SWAV

SourceMods:

/glade/u/home/oleson/run_hist_1850_files/SPINUP/casefiles_cecile/SourceMods/ctsm5.1.dev120/src.clm/*

CLM namelists:

use_init_interp = .true.
check_finidat_year_consistency = .false.
fsurdat = '/glade/work/slevis/git/mksurfdata_toolchain/tools/mksurfdata_esmf/surfdata_ne30np4.pg3_SSP5-8.5_78pfts_CMIP6_1850-2100_c230227.nc'
flanduse_timeseries = '/glade/work/slevis/git/mksurfdata_toolchain/tools/mksurfdata_esmf/landuse.timeseries_ne30np4.pg3_SSP5-8.5_78_CMIP6_1850-2100_c230227.nc'

CAM namelists:

mfilt    =       0,       5,     20,      40,      12,       1,      1
nhtfrq              =       0,     -24,    -24,      -3,       0,       0,      0
ndens               =       2,       2,      2,       2,       2,       1,      2
interpolate_output  =  .true.,  .true., .true., .false., .false., .false.,  .true.
interpolate_nlat    =     192,     192,    192,     192,     192,     192,   192
interpolate_nlon    =     288,     288,    288,     288,     288,     288,   288 

fexcl1 = ' '  

empty_htapes = .true.

fincl1 = 'ACTNI', 'ACTNL', 'ACTREI', 'ACTREL', 'AODDUST', 'AODVIS', 'AODVISdn','BURDENBC', 'BURDENDUST', 'BURDENPOM', 'BURDENSEASALT', 
'BURDENSO4', 'BURDENSOA', 'CAPE', 'CCN3', 'CDNUMC', 'CH4', 'CLDHGH', 'CLDICE', 'CLDLIQ', 'CLDLOW', 'CLDMED', 'CLDTOT', 'CLOUD', 'CMFMC_DP', 
'CT_H2O', 'DCQ', 'DQCORE', 'DTCOND', 'DTCORE', 'DTV', 'EVAPPREC', 'EVAPSNOW', 'FCTI', 'FCTL', 'FICE', 'FLDS', 'FLNS', 'FLNSC', 'FLNT', 'FLNTC', 'FLUT', 
'FREQZM', 'FSDS', 'FSDSC', 'FSNS', 'FSNSC', 'FSNT', 'FSNTC', 'FSNTOA', 'ICEFRAC', 'LANDFRAC', 'LHFLX', 'LWCF', 'MPDICE', 'MPDLIQ', 'MPDQ', 'MPDT', 
'OCNFRAC', 'OMEGA', 'OMEGA500', 'PBLH', 'PHIS', 'PINT', 'PMID', 'PRECC', 'PRECL', 'PRECSC', 'PRECSL', 'PRECT', 'PS', 'PSL', 'PTEQ', 'PTTEND', 'Q', 
'QFLX', 'QRL', 'QRS', 'QTGW', 'RCMTEND_CLUBB', 'RELHUM', 'RVMTEND_CLUBB', 'SHFLX', 'SOLIN', 'SST', 'STEND_CLUBB', 'SWCF', 
'T', 'TAUX', 'TAUY', 'TFIX', 'TGCLDIWP', 'TGCLDLWP', 'TMQ', 'TREFHT', 'TS', 'TTGW', 'U', 'U10', 'UBOT', 'UTGWORO', 'UTGW_TOTAL', 
'V', 'VBOT', 'VTGWORO', 'VTGW_TOTAL', 'WPRTP_CLUBB', 'WPTHLP_CLUBB', 'Z3', 'ZMDQ', 'ZMDT', 'N2O', 'CO2','CFC11','CFC12', 
'CLD_MISR','FISCCP1_COSP','CLD_CAL','CLD_MISR','CLDTOT_CAL','CLDHGH_CAL', 'CLDMED_CAL','CLDLOW_CAL','CLMODIS', 'AODVISdn'

fincl3 = 'PRECT', 'PRECC', 'FLUT', 'U850', 'U200', 'V850', 'V200', 'OMEGA', 'PSL'

fincl4 =  'PRECC','PRECL'

fincl5 = 'Uzm','Vzm','Wzm','THzm', 'VTHzm','WTHzm','UVzm','UWzm'

fincl6 =  'WV_phBF','WL_phBF','WI_phBF','SE_phBF','KE_phBF',
'WV_phBP','WL_phBP','WI_phBP','SE_phBP','KE_phBP',
'WV_phAP','WL_phAP','WI_phAP','SE_phAP','KE_phAP',
'WV_phAM','WL_phAM','WI_phAM','SE_phAM','KE_phAM',
'WV_dyBF','WL_dyBF','WI_dyBF','SE_dyBF','KE_dyBF',
'WV_dyBP','WL_dyBP','WI_dyBP','SE_dyBP','KE_dyBP',
'WV_dyAP','WL_dyAP','WI_dyAP','SE_dyAP','KE_dyAP',
'WV_dyAM','WL_dyAM','WI_dyAM','SE_dyAM','KE_dyAM',
'WV_dBF','WL_dBF','WI_dBF','SE_dBF','KE_dBF',
'WV_dED','WL_dED','WI_dED','SE_dED','KE_dED',
'WV_dAD','WL_dAD','WI_dAD','SE_dAD','KE_dAD',
'WV_dAF','WL_dAF','WI_dAF','SE_dAF','KE_dAF',
'WV_dBD','WL_dBD','WI_dBD','SE_dBD','KE_dBD',
'WV_dAR','WL_dAR','WI_dAR','SE_dAR','KE_dAR',
'WV_dAH','WL_dAH','WI_dAH','SE_dAH','KE_dAH',
'WV_dBH','WL_dBH','WI_dBH','SE_dBH','KE_dBH',
'WV_dCH','WL_dCH','WI_dCH','SE_dCH','KE_dCH',
'WV_dAS','WL_dAS','WI_dAS','SE_dAS','KE_dAS',
'WV_dBS','WL_dBS','WI_dBS','SE_dBS','KE_dBS',
'EFIX'

fincl7= 'AQSO4_H2O2','AQSO4_O3', 'bc_a1', 'bc_a4', 'dst_a1', 'dst_a2', 'dst_a3', 'ncl_a1',
'ncl_a1', 'ncl_a2', 'ncl_a3', 'pom_a1', 'pom_a4', 'so4_a1', 'so4_a2', 'so4_a3',
'soa_a1', 'num_a1', 'num_a2', 'num_a3', 'num_a4',
'bc_a1SFWET', 'bc_a4SFWET', 'dst_a1SFWET', 'dst_a2SFWET', 'dst_a3SFWET', 'ncl_a1SFWET',
'ncl_a2SFWET', 'ncl_a3SFWET', 'pom_a1SFWET', 'pom_a4SFWET', 'so4_a1SFWET', 'so4_a2SFWET', 'so4_a3SFWET', 'soa_a1SFWET',
'soa_a2SFWET', 'bc_c1SFWET', 'bc_c4SFWET', 'dst_c1SFWET', 'dst_c2SFWET', 'dst_c3SFWET', 'ncl_c1SFWET', 'ncl_c2SFWET',
'ncl_c3SFWET', 'pom_c1SFWET', 'pom_c4SFWET', 'so4_c1SFWET', 'so4_c2SFWET', 'so4_c3SFWET', 'soa_c1SFWET', 'soa_c2SFWET',
'bc_a1DDF', 'bc_a4DDF', 'dst_a1DDF', 'dst_a2DDF', 'dst_a3DDF', 'ncl_a1DDF', 'ncl_a2DDF', 'ncl_a3DDF',
'pom_a1DDF', 'pom_a4DDF', 'so4_a1DDF', 'so4_a2DDF', 'so4_a3DDF', 'soa_a1DDF', 'soa_a2DDF',
'so4_a1_CLXF', 'so4_a2_CLXF', 'SFbc_a4', 'SFpom_a4', 'SFso4_a1', 'SFso4_a2',
'so4_a1_sfgaex1', 'so4_a2_sfgaex1', 'so4_a3_sfgaex1', 'soa_a1_sfgaex1', 'soa_a2_sfgaex1',
'SFdst_a1','SFdst_a2', 'SFdst_a3', 'SFncl_a1', 'SFncl_a2', 'SFncl_a3',
'num_a2_sfnnuc1'

phys_grid_ctem_nfreq=-6
phys_grid_ctem_zm_nbas=120
phys_grid_ctem_za_nlat=90

ncdata = '/glade/p/cesm/amwg_dev/juliob/FWsc_ne30pg3_58L_GRID_48_taperstart10km_lowtop_BL10_v3_beta1p75_Top_43km.nc'

clubb_l_predict_upwp_vpwp=.true. 
clubb_l_mono_flux_lim_um   = .true.  
clubb_l_mono_flux_lim_vm   = .true.  
clubb_c_uu_shr = 0.1
clubb_c7=0.1

dust_emis_fact         = 0.80D0

Case directory: Locally (if still available): /glade/p/cesmdata/cseg/runs/cesm2_0/f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001

On github: https://github.com/NCAR/amwg_dev/tree/f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001

Sandbox: Locally (if still available): /glade/work/hannay/cesm_tags/cam6_3_108

On github: hash: (https://github.com/ESCOMP/CAM/tree/cam6_3_108)

Diagnostics: ADF diags (if available) https://webext.cgd.ucar.edu/FLTHIST/f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001/atm/

Contacts: @wwieder, @swrneale, @PeterHjortLauritzen, @JulioTBacmeister, @cacraigucar, @olyson, @adamrher

[cecilehannay]

This run crashed in the land initialization is crashing with the error:

771:MPT:     at /glade/work/hannay/cesm_tags/cam6_3_107/components/clm/src/main/ncdio_pio.F90.in:1512
771:MPT: #21 0x00000000021df00d in initinterpmod_mp_initinterp_ ()
771:MPT:     at /glade/work/hannay/cesm_tags/cam6_3_107/components/clm/src/init_interp/initInterp.F90:1041
771:MPT: #22 0x000000000210a4aa in clm_initializemod_mp_initialize2_ ()

The last thing that happens in the land log is:

 Interpolating: T10_VALUE => T10_VALUE
 Interpolating: T10_NSTEPS => T10_NSTEPS
 Interpolating: SOIL10_VALUE => SOIL10_VALUE

In case this helping: in a previous successful run (without the clm mods), the variables that were interpolated after that were:

 Interpolating: SOIL10_NSTEPS => SOIL10_NSTEPS
 Interpolating: TDM5_VALUE => TDM5_VALUE
 Interpolating: TDM5_NSTEPS => TDM5_NSTEPS
 Interpolating: SNOW_5D_VALUE => SNOW_5D_VALUE
 Interpolating: SNOW_5D_NSTEPS => SNOW_5D_NSTEPS
 Interpolating: BTRANAV_VALUE => BTRANAV_VALUE
 Interpolating: BTRANAV_NSTEPS => BTRANAV_NSTEPS

@wwieder and @olyson: any advice?

[olyson]

Ok, I suspect that initial file is too old to work with new code because of this issue where SOIL10_VALUE changed from a pft-level to a column-level variable:

https://github.com/ESCOMP/CTSM/issues/1708

So I think that you'll have to go back to the initial files you've used previously. Since I recently learned this is an SP configuration, I will say that I don't think the initial file is quite as critical since we don't need spunup initial carbon and nitrogen states and the remaining longer time-scale states (e.g., soil temperature and moisture) will equilibrate fairly quickly.

[adamrher]

I agree with @olyson; let's just use the inic we usually use, as this is just an SP run.

It has its advantage in that we can isolate changes to the change in physics+surface dataset, and not due to differences in inic.

[cecilehannay]

@justin-richling: could you run the ADF for:

• f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001 versus f.cam6_3_105.FLTHIST_v0a.ne30.001

Please add the variables FSNTC and FLNTC. and use the same set of years for both runs. Thanks

[justin-richling]

ADF diags are here

[adamrher]

@JulioTBacmeister, these clear sky fluxes over land are an improvement, right?

[JulioTBacmeister]

They certainly seem to be. We wanted a decrease in FSNTC and that seems to have occurred. I am trying to look at more diagnostics but I guess the power down has shut me out. Anyone recall what FLNTC was with CLM5.1. I'll look directly on Casper later.

[olyson]

Land diags are here There are increases in surface albedo mainly in forested regions (see ASA in set2 contour plots) presumably due to the changes in leaf optical properties in CTSM5.1. These correspond to the regions with decreases in FSNTC (I'm assuming the sign conventions make sense here).

[justin-richling]

Updated ADF diags here

[billsacks]

@cecilehannay @cacraigucar @adamrher @olyson and others: If you use ctsm5.1.dev122 or later you shouldn't need the SnowHydrologyMod.F90 SourceMods (changing custom_rel_epsilon): that CTSM tag resolves the issue in what I hope is a more robust way.

[cacraigucar]

@billsacks - I see that ctsm5.1.dev122 is scheduled to go into the cesm2_3_alpha12g which is the tag that also includes CAM's energy changes. Removing yet another set of SourceMods is good!

[adamrher]

I just spoke w/ @bstephens82 about a run he did with just the new CLM51 physics (not the new surface datasets), and there is no such change in FSNTC:

Suggesting that the albedo changes @olyson finds are likely from the new surface datasets. Ben is re-running with the new surface datasets to verify this.

The implications of this are: if we want to use the new CLM52 surface datasets in the coupled sims, we will need to SourceMod in the ability to run these in a CLM51 tag. It's my understanding we are ~months out from a proper CLM52 tag that wouldn't require SourceMods to run these. I think we need to make sure everyone is aware of this inconvenience, as well as an update on the progress of a CLM52 tag, at the co-chairs mtg tmrw.

[olyson]

Is this the case:

/glade/u/home/stepheba/cam107_newland/cime/scripts/dflt107_newland

If so, that is still using the CLM50 model:

2000_CAM60_CLM50%SP_CICE%PRES_DOCN%DOM_MOSART_SGLC_SWAV_SESP

[bstephens82]

Is this the case:

/glade/u/home/stepheba/cam107_newland/cime/scripts/dflt107_newland

If so, that is still using the CLM50 model:

2000_CAM60_CLM50%SP_CICE%PRES_DOCN%DOM_MOSART_SGLC_SWAV_SESP

Yes that's the case, although I did update Externals.cfg with ctsm5.1.dev122, and in components/clm/doc/ChangeLog it has dev122 as well. I am also using SourceMods and namelist settings from @adamrher. Is it still not doing what we want?

[bstephens82]

Ok I spoke to Cheryl and hopefully have it set up correctly now. I'm using the longname compset @olyson mentioned above except with CLM51, I've updated Externals.cfg with ctsm5.1.dev122, and I'm using the SourceMods and user_nl_clm given above.

[adamrher]

@bstephens82, no, it is not doing what you want. You need to explicitly set it to CLM51 physics in the long name compset. Use the long name compset that Cecile posted at the top of this thread (specifically, the CLM51%SP part).

@olyson would you expect the albedo changes to be from the CLM51 physics or the surface datasets?

[edit - just seeing Ben's response. That's right.]

[olyson]

@adamrher , Most of the albedo changes are coming from changes in the plant functional type optical properties which are prescribed in the CLM parameter file. A new parameter file containing those new optical properties is invoked by the CLM51 compset (in lnd_in):

paramfile = '/glade/p/cesmdata/cseg/inputdata/lnd/clm2/paramdata/ctsm51_params.c211112.nc'

[adamrher]

Got it. Thanks for also catching Ben's run. We should expect to see the albedo changes with the new CLM51 physics, then.

Ben let us know if your new runs confirm this.

[brianpm]

I'm running into an issue with the output of this run. For some reason CLDLOW_CAL has very large negative numbers where there should be missing values. Example using the first file: /glade/scratch/hannay/archive/f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001/atm/hist/f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001.cam.h0.1979-01.nc

I'm not sure if this is only happening in this run... I'm going to go look at others.

[brianpm]

Update -- It is not just this run:

First h0 file from each of these runs indicate there are a bunch of negative numbers. There are also missing values. Could this be an issue with interpolation?

[case: f.cam6_3_101.FLTHIST_v0a.ne30.002] less than zero: 1146; missing: 1849
[case: f.cam6_3_101.FLTHIST_v0a.ne30.003] less than zero: 1108; missing: 1824
[case: f.cam6_3_102.FLTHIST_v0a.ne30.001] less than zero: 1108; missing: 1824
[case: f.cam6_3_105.FLTHIST_v0a.ne30.001] less than zero: 1108; missing: 1824
[case: f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001] less than zero: 1107; missing: 1822
[case: f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001] less than zero: 1078; missing: 1821
[case: f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw_ebudget.001] less than zero: 1099; missing: 1746
[case: f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw_ubcF.001] less than zero: 1079; missing: 1819
[case: f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw_lcp-moistF.001] less than zero: 1090; missing: 1819
[case: f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1_ebudget.001] less than zero: 1146; missing: 1849
[case: f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1.001] less than zero: 1120; missing: 1831
[case: f.cam6_3_107.FLTHIST_v0a.ne30.clm5_1_ebudget_tuning1.001] less than zero: 1090; missing: 1819

[adamrher]

I believe this is known issue -- the interpolate_output functionality for the se-dycore doesn't retain missing values as missing values -- that we decided was not worth the trouble to fix. if the missing values are fixed in time, than a quick post-process correction should resolve the issue. @nusbaume can chime in as he knows more about this issue.

[nusbaume]

Hi All,

I believe the issue @brianpm is running into is due to a mis-match between what COSP and the interpolation code are assuming for fillvalue "values". I believe I have found a solution for it, but am planning to run a couple of tests first. Stay tuned!