merge to clm > r17x-ish and stable cime tag

[bandre-ucar]

Summary of Issue:

Merge to a more recent clm trunk tag based on clm4_5_x > r17x-ish. This will include critical bugfixes for clm, new but stable cime user interface scripts and a better starting point for clm5 - ED integration.

Code review and testing needed for:

• port to LBL, LANL machines

  Significant user interfaces changes

• Case scripts are now 'case.build' instead of '${CASE_NAME}.build'

[bandre-ucar]

Work taking place on:

andre-ed-clm-16x

ed-clm-master changeset: c3a1f92

Testing: for update through r150 were 'ok'. expected failures for f09 and f19 restarts, #14. answer changing, no baseline comparison.

[bandre-ucar]

Testing for r159 - massive failures for ed tests related to pio. Expected because of problems with ed/pio in clm trunk. Going to continue merging to current r174. If problems aren't fixed, then backtrack to r159-ish or fix on head....?

[bandre-ucar]

TODO(bja): Check CNWoodProducts change for cnveg nitrogen state intent from Stef's branch is in new CNProducts?

[bandre-ucar]

Testing on yellowstone with c3a1f92 / r175 just to see where things stand.

[bandre-ucar]

The clm-ed parameter file needs to be merged with the standard clm parameter file. Merged clm_params_ed.c151027.nc with clm_params.c160225.nc to generate clm_params_ed.c160225.nc. The resulting parameter set needs science review. slatop had minor conflicts.

TODO(bja, 20160412) add clm_params_ed.c160225.nc to input data repo!

[ckoven]

Ben -- what are the differences in the variable lists between these PFT parameter files?

[bandre-ucar]

The major difference seemed to be the addition of flexcn/luna/fun parameters. You can run see the files and run ncdump on yellowstone /glade/p/cesm/cseg/inputdata/lnd/clm2/paramdata/ or I can add the file to the svn input repo so you can download it.

[ckoven]

But these PFT parameters ought not to really apply to this model, since it currently has neither LUNA nor FUN nor FlexCN in it. So are the new variables even read when ED is active?

[ckoven]

Maybe an alternate solution is to have two possible sets of PFT files: one for the CN veg model and one for the ED veg model? It seems like this would be required during interfacing anyway (since, e.g. CLM and ACME non-demographic veg models will definitely have different PFT parameters), and would reduce any confusion about which PFT parameters are in use in a given model configuration?

[bandre-ucar]

All the parameters are required to be in the file even, if they aren't used.

I think we will definitely want to stop reusing the clm parameter file and create a separate file for ED.

[rosiealice]

seems like the parameter file strategy might be a thing we could discuss tomorrow, if we have any time left over?

On 13 April 2016 at 12:38, Ben Andre notifications@github.com wrote:

All the parameters are required to be in the file even, if they aren't used.

I think we will definitely want to stop reusing the clm parameter file and create a separate file for ED.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/NGEET/ed-clm/issues/46#issuecomment-209587841

---

Dr Rosie A. Fisher

Terrestrial Sciences Section Climate and Global Dynamics National Center for Atmospheric Research 1850 Table Mesa Drive Boulder, Colorado, 80305 USA. +1 303-497-1706

http://www.cgd.ucar.edu/staff/rfisher/

[bandre-ucar]

Failing restart tests with the r175 based ed:

FAIL ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest.clm2.h0.nc : test compare clm2.h0 (.base and .rest files) 
FAIL ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest.cpl.hi.nc : test compare cpl.hi (.base and .rest files) 
FAIL ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest : test functionality summary (ERS_test) 

FAIL ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_intel.clm-edTest.clm2.h0.nc : test compare clm2.h0 (.base and .rest files) 
FAIL ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_intel.clm-edTest.cpl.hi.nc : test compare cpl.hi (.base and .rest files) 
FAIL ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_intel.clm-edTest : test functionality summary (ERS_test) 

Bisecting through the history:

r150 - f10 pass
r151 - f10 pass
r152 - f10 pass
r153 - f10 fail
r155 - f10 fail
r158 - f10 fail
r160 - f10 fail
r175 - f10 fail

ed-clm at r153, 56dfb030, introduced the restart failure at f10 and f45.

From the clm short log:

clm4_5_6_r153    sacks 11/17/2015 Fix snow cover fraction bug

r153 is a very small delta. The CanopyHydrology change:

              !======================  FSCA PARAMETERIZATIONS  ======================
              ! fsca parameterization based on *changes* in swe
              ! first compute change from melt during previous time step
-             if(snowmelt(c) >= 0._r8) then
+             if(snowmelt(c) > 0._r8) then

                 smr=min(1._r8,(h2osno(c))/(int_snow(c)))

When this is reverted, i.e. '>' back to '>=', the failing ED tests pass. Have not yet investigated why....

[bandre-ucar]

All the variables related to the frac_sno already appear to be on the restart file.

Restart is not bit for bit at a single point in the antarctic....

ED interacts through frac_sno_eff.

[bandre-ucar]

As a sanity check, I tried the other r153, 56dfb03, delta from SnowHydrology:

components/clm/src/biogeophys/SnowHydrologyMod.F90
@@ -625,10 +625,12 @@ subroutine SnowCompaction(bounds, num_snowc, filter_snowc, &
                       ddz3 = max(0._r8,min(1._r8,(swe_old(c,j) - wx)/wx))

                       ! 2nd term is delta fsno over fsno, allowing for negative values for ddz3
-                      wsum = sum(h2osoi_liq(c,snl(c)+1:0)+h2osoi_ice(c,snl(c)+1:0))
-                      fsno_melt = 1. - (acos(2.*min(1._r8,wsum/int_snow(c)) - 1._r8)/rpi)**(n_melt(c))
-
-                      ddz3 = ddz3 - max(0._r8,(fsno_melt - frac_sno(c))/frac_sno(c))
+                      if((swe_old(c,j) - wx) > 0._r8) then
+                         wsum = sum(h2osoi_liq(c,snl(c)+1:0)+h2osoi_ice(c,snl(c)+1:0))
+                         fsno_melt = 1. - (acos(2.*min(1._r8,wsum/int_snow(c)) - 1._r8)/rpi)**(n_melt(c))
+                         
+                         ddz3 = ddz3 - max(0._r8,(fsno_melt - frac_sno(c))/frac_sno(c))
+                      endif
                       ddz3 = -1._r8/dtime * ddz3
                    else
                       ddz3 = - 1._r8/dtime * max(0._r8,(frac_iceold(c,j) - fi)/frac_iceold(c,j))

This does NOT allow the exact restart tests to pass.

Testing done with: ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_intel.clm-edTest

[ckoven]

@bandre-ucar so did this isolation of the code that breaks restart allow you to solve the issue? If not, should we reach out to someone with specific knowledge of the snow code to get this sorted out? thanks-

[bandre-ucar]

The problem isn't with snow, it's with ED, either it's interaction with snow or this is just highlighting the generally poor restart capability in ED (see #14 and #43).

I have a work around, so I'm going to use it, and move on. This workaround is in the CLM code, and will be unacceptable to put back into mainline CLM, so this will have to be resolved at some point. Further discussion and work on this issue should take place in #74.

[bandre-ucar]

Merging from r175-r180 results in runtime failures.

RUN ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest.06131631-edi 

This seems to happen for all ERS_D tests. There is no useful information in the logs and no core files.

Testing intermediate clm trunk tags:

175 - 06f46199 - runs 176 - f40a5e86 - runs 177 - a2bf3042 - runs 178 - 7107a26a - dies at runtime 180 - 6dfd39d2 - dies at runtime

7107a26a has the following error in the cesm.log file:

   1:Create file ./ed-clm-i46-6dfd39d2.clm2.r.0001-01-06-00000.nc 65536
   1:Abort with message NetCDF: String match to name in use in file in file /glade/scratch/andre/ed-clm-i46-6dfd39d2/bld/intel/mpich2/nodebug/nothreads/pio/src/clib/pio_nc.c at line 393

There is no useful information in the stack trace and no core files.

The last line in the land log file is:

 hist_htapes_wrapup : history tape            1 : no open file to close
 writing restart file ./ed-clm-i46-6dfd39d2.clm2.r.0001-01-06-00000.nc
  for model date = 0001-01-06-00000                

 restFile_open: writing restart dataset at 
 ./ed-clm-i46-6dfd39d2.clm2.r.0001-01-06-00000.nc at nstep =          240

r178 is:

Tag name:  clm4_5_8_r178
 Originator(s):  sacks (Bill Sacks)
 Date: Sun Apr 17 19:28:55 MDT 2016
 One-line Summary: Remove some consistency checks, and merge crop_prog with use_crop in code

There was also a problem with this merge. Need to investigate further if the runtime failure is a result of the merge or r178.

[bandre-ucar]

Reran the failing case in debug mode in the debugger. Error is:

Abort with message NetCDF: String match to name in use in file in file /glade/scratch/andre/ed-clm-i46-7107a26a/bld/intel/mpich2/debug/nothreads/pio/src/clib/pio_nc.c at line 393

From the stack, the variable is ED_GDD0_VALUE coming out of accumulMod::accumulrest line 596. Next TODO item is investigate why ED_GDD0 appears to be added twice.

[ckoven]

this sounds like a conflict with the phenology refactoring. i deleted any interaction between fates and the accum machinery in that so sounds like the issue is there.

Charlie

Sent from a mobile device.

On Jun 16, 2016, at 4:38 PM, Ben Andre notifications@github.com wrote:

Reran the failing case in debug mode in the debugger. Error is:

Abort with message NetCDF: String match to name in use in file in file /glade/scratch/andre/ed-clm-i46-7107a26a/bld/intel/mpich2/debug/nothreads/pio/src/clib/pio_nc.c at line 393 From the stack, the variable is ED_GDD0_VALUE coming out of accumulMod::accumulrest line 596. Next TODO item is investigate why ED_GDD0 appears to be added twice.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[bandre-ucar]

Yep, it looks like I didn't handle the merge conflicts correctly in TemperatureType.F90. Running the full test suite on the fixed version of the code.

[bandre-ucar]

I've merged the fix from @rosiealice for the ed r153 restart problem into my branch for the clm trunk merge. The current status of the branch is: based on clm-ed master at c3a1f922, up to date with clm trunk r180. All tests pass. Next step is a series of merges to bring it up to the current clm-ed master.

[rosiealice]

Nice job Ben!

On 22 June 2016 at 14:04, Ben Andre notifications@github.com wrote:

I've merged the fix from @rosiealice https://github.com/rosiealice for the ed r153 restart problem into my branch for the clm trunk merge. The current status of the branch is: based on clm-ed master at c3a1f92 https://github.com/NGEET/ed-clm/commit/c3a1f922aa8b5b9f02e22071625f7767eb91d353, up to date with clm trunk r180. All tests pass. Next step is a series of merges to bring it up to the current clm-ed master.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/ed-clm/issues/46#issuecomment-227860477, or mute the thread https://github.com/notifications/unsubscribe/AMWsQ9s4fIxRsiToDhMGjwkMEjTLAUjSks5qOZVqgaJpZM4IBO7B .

---

Dr Rosie A. Fisher

Terrestrial Sciences Section Climate and Global Dynamics National Center for Atmospheric Research 1850 Table Mesa Drive Boulder, Colorado, 80305 USA. +1 303-497-1706

http://www.cgd.ucar.edu/staff/rfisher/

[bandre-ucar]

clm-ed master commits that need to be merged:

c3a1f922 - done, all tests pass 90c37589 - done, issue template only, no tests run 8740a1a3 - done, all tests pass, merge conflicts in machines 19fe5678 - done, all tests pass d8a9ee50 - done, all tests pass c23cf02e - done, all tests pass. conflicts require scientific review of commit eebdf4d by @ckoven 534d1529 - done, all tests pass 5d120663 - machines update, single test passed. 763a7226 - done, all tests pass 89b8709b - done, all tests pass 584f2a14 - done, all tests pass 0dec8c38 - done, doc only no testing. f8817210 - done, all tests pass 0471ef9e - done, all tests pass 5421da59 - done, all tests pass 1aaba897 - done, all tests pass 94118a5b - done, all tests pass 1fc68114 - done, all tests pass 57c533c4 - done, all tests pass 18613d14 - done, all tests pass; pio2 issue with r180, resolved by merged branch to clm-r181. columnization issues resolved by @rgknox. (runtime failures in restart tests related to columnization. review of f7c3eee @rgknox) c0654db - done, introduced new expected failure in tests, see #88 bd4719b - done, all ed and clm_short tests pass f8e6313 - done, all ed and clm_short tests pass

[ckoven]

@bandre-ucar,

I'm looking through the commits to eebdf4d as you suggested. a couple things:

https://github.com/NGEET/ed-clm/commit/eebdf4dc5a4a55edefd09a74b95b4597f77d4181#diff-495103cd15f8049be0952b4736d75bf5R2407 -- the logic here ought to be related to use_vertsoilc rather than use_century_decomp. if in SP mode, then none of this ought to be on

http://github.com/NGEET/ed-clm/commit/eebdf4dc5a4a55edefd09a74b95b4597f77d4181#diff-86b7b164f9607a4d86d626947a92af4bR95 -- only the cnveg_carbonstate_type and cnveg_carbonflux_type are actually used in EDBGCDynMod.F90, so you could delete the occurrences of cnveg_state_inst (and actually cnveg_nitrogenstate_type and cnveg_nitrogenflux_type too) from EDBGCDynMod.F90, but agreed that this could be interfaced in a better way.

[bandre-ucar]

@ckoven the use_century_decomp in buildnamelist is necessary because of a runtime error associated with setting i_cwd in clm_varpar. using vertsoilc won't fix the problem. Please create a branch and address any issues that you see and I will fold them into my branch.

[bandre-ucar]

@rgknox I've pushed the columnization merge to my fork. You can create a branch off of my merge at f7c3eee0 and work from there, or you can create a branch from eb51b2a2 and remerge 18613d1.

   2: fcolumn(1) in this clump, points to the first column of the clump
   2: but the assumption on first cohort index does not jive
   2: ENDRUN:
   2: ERROR in EDRestVectorMod.F90 at line 1516                                 

[rgknox]

ok, looking into this now

On Wed, Jun 29, 2016 at 9:20 AM, Ben Andre notifications@github.com wrote:

@rgknox https://github.com/rgknox I've pushed the columnization merge to my fork. You can create a branch off of my merge at f7c3eee https://github.com/NGEET/ed-clm/commit/f7c3eee0f139d2ec6e43ec9221efb53faafe5a64 and work from there, or you can create a branch from eb51b2a https://github.com/NGEET/ed-clm/commit/eb51b2a29d514240db76a956589aad625933f0d9 and remerge 18613d1 https://github.com/NGEET/ed-clm/commit/18613d14f089076ee18b5ada73715ee85688845b .

2: fcolumn(1) in this clump, points to the first column of the clump 2: but the assumption on first cohort index does not jive 2: ENDRUN: 2: ERROR in EDRestVectorMod.F90 at line 1516

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/ed-clm/issues/46#issuecomment-229408984, or mute the thread https://github.com/notifications/unsubscribe/AFnnjDjIK3WKCp7E8j4ARPo0pWTQjj9mks5qQps0gaJpZM4IBO7B .

[rgknox]

I think the issue is in subgridmod.

The columnization stuff started with a simple assumption that IO space would be made for cohorts on all columns, which is not a great way to do it, and needs to be changed, but it was how I coded it.

In your branch Ben, the cohort space is calculated in subgrid_get_info_natveg, and as you can see ncohort is defined as the number of cohorts times the number of natural vegetation columns.

The general fix is to make ncohorts = ncols * cohorts_per_col after the accumulators in subgrid_get_gcell_info have gone through all of those sub-calls, ie after the crop (which is last).

so.. I think something like this should work:

 call subgrid_get_info_crop(gi, npatches_temp, ncols_temp, nlunits_temp)
    call accumulate_counters()

    ncohorts = ncols*cohorts_per_col

[rgknox]

I will create a new issue that states the IO space for cohorts needs to be linked to the number of natural vegetation columns and not total columns.

[ckoven]

@bandre-ucar, ok I made a new commit that is branched off of yours that ought to resolve the i_cwd issue. the commit is 853f0066. could you test and confirm that it does? thanks.

The commit also reduces by 1 the number of required public members of the cn_vegetation_type in CNVegetationFacade.F90, but there are still 2 left. I guess we ought to discuss the strategy for that but there is a lot going on in that module though that I don't understand...

[bandre-ucar]

@rgknox I made the change you proposed and now I get another runtime error:

   0:forrtl: severe (408): fort: (2): Subscript #1 of the array BALIVE has value 1 which is greater than the upper bound of 0

   0:cesm.exe           0000000001413FD4  edrestvectormod_m        1564  EDRestVectorMod.F90

[rgknox]

not sure what is going on there @bandre-ucar. It seems as though there is no allocated space in the restart vectors for cohorts (size 0?).

If it wants to use global cohort index 1, it seems as though we are on the first column of the the first grid-cell. In this case the restart vector this%balive() should be allocated from 1:num_cohort_per_column (which should also be bounds%begcohort:bounds%endcohort.

If it is passing the logic trap (keyword "jive") where it was previously failing, then it seems that bounds%begCohort is non-zero.

I'm trying to get this branch working on my machines right now, so hopefully I will be able to do some testing soon and reproduce this error.

I think the next thing I would want to look at is the allocation bounds for balive (~ line 339) as they appear for each column. bounds%begcohort should = (bounds%begc-1)*num_cohort_per_col + 1

[bandre-ucar]

@ckoven Your branch dies at run time. There were several errors in the original merge related to the namelist, I may have mislead you about the error before.

Here's part of the stack trace from the core dump:

#5  <signal handler called>
#6  0x0000000001afdffe in soilbiogeochemdecompcascadebgcmod::decomp_rate_constants_b
gc (bounds=..., num_soilc=8, filter_soilc=..., canopystate_inst=..., soilstate_inst=
..., temperature_inst=..., ch4_inst=..., soilbiogeochem_carbonflux_inst=...) at /gla
de/p/work/andre/ed/ed-clm-ckoven/components/clm/src/soilbiogeochem/SoilBiogeochemDec
ompCascadeBGCMod.F90:1012
#7  0x00000000011f1eb4 in edbgcdynmod::edbgcdyn (bounds=..., num_soilc=8, filter_soi
lc=..., num_soilp=16, filter_soilp=..., num_pcropp=0, filter_pcropp=..., doalb=.FALSE., cnveg_carbonflux_inst=..., cnveg_carbonstate_inst=..., ed_clm_inst=..., soilbiogeochem_carbonflux_inst=..., soilbiogeochem_carbonstate_inst=..., soilbiogeochem_state_inst=..., soilbiogeochem_nitrogenflux_inst=..., soilbiogeochem_nitrogenstate_inst=..., c13_soilbiogeochem_carbonstate_inst=..., c13_soilbiogeochem_carbonflux_inst=..., c14_soilbiogeochem_carbonstate_inst=..., c14_soilbiogeochem_carbonflux_inst=..., atm2lnd_inst=..., waterstate_inst=..., waterflux_inst=..., canopystate_inst=..., soilstate_inst=..., temperature_inst=..., crop_inst=..., ch4_inst=...) at /glade/p/work/andre/ed/ed-clm-ckoven/components/clm/src/biogeochem/EDBGCDynMod.F90:186

decomp_depth_efolding is zero and you are getting a divide by zero. decomp_depth_efolding is zero by default, so it isn't being read from the namelist. Verified that the relevant section of the namelist is empty:

 soil_layerstruct = '10SL_3.5m'
 use_bedrock = .false.
 use_century_decomp = .true.
 use_cn = .false.
 use_crop = .false.
 use_dynroot = .false.
 use_ed = .true.
 use_ed_spit_fire = .true.
 use_fertilizer = .false.
 use_fun = .false.
 use_grainproduct = .false.
 use_lai_streams = .false.
 use_lch4 = .false.
 use_luna = .false.
 use_nitrif_denitrif = .false.
 use_vertsoilc = .true.
/
...
...
...
&bgc_shared
/

That section of the namelist is empty because because it is only added when bgc_mode != sp. use_cn == false so we are in sp mode. That is why I changed the logic to trigger setting decomp_depth_efolding based on another variable.

[rgknox]

@bandre-ucar , what test or setup will reproduce the error you mentioned above with balive bounds?

[bandre-ucar]

@rgknox You can test with:

ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest

[ckoven]

@bandre-ucar I am confused about which options are triggering the error. this is happening in SP mode or in ED mode?

[bandre-ucar]

@ckoven The default conditions used by all the ed tests appear to be ed mode with sp. You can recreate it with:

cd cime/scripts
./create_test -testname ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest -testid junk-r180

The snippet of the lnd_in above came from that test.

[ckoven]

@bandre-ucar ok so I think the issue is the scripting logic. "ed" ought to be a value for bgc_mode as ED and SP are mutually exclusive possibilities. The old logic may have allowed that since in both cases the soil biogeochemistry is off, but now the soil biogeochemistry is on (or a subset of it anyway) whenever ED is on, so the mutual incompatibility of those is crashing the model. I've made a new commit, f47cd81, which I think sets the switches the way I think they ought to be set.

I tried running the test on yellowstone but am running into other issues -- could you try testing the new commit? thanks

[bandre-ucar]

@ckoven ok, testing.

[bandre-ucar]

@ckoven You have a typo in the xml file you modified:

$ git log -1
commit f47cd8179421049e7e31cf60cd60b4a2b1626e24
Author: Charlie Koven <cdkoven@lbl.gov>
Date:   Thu Jun 30 13:10:44 2016 -0700

    set bgc_mode to 'ed' whenever use_ed = .true. to avoid the old default behavior of bgc_mode = 'sp' when ED was on which doesn't make sense
$ git diff
diff --git a/components/clm/bld/namelist_files/namelist_defaults_clm4_5.xml b/components/clm/bld/namelist_files/namelist_defaults_clm4_5.xml
index 75d2cb6..0a4ab56 100644
--- a/components/clm/bld/namelist_files/namelist_defaults_clm4_5.xml
+++ b/components/clm/bld/namelist_files/namelist_defaults_clm4_5.xml
@@ -1802,7 +1802,7 @@ lnd/clm2/surfdata_map/surfdata_ne120np4_78pfts_simyr1850_c160216.nc</fsurdat>
 <use_ed_spit_fire use_ed=".true.">.true.</use_ed_spit_fire>
 <use_lch4 use_ed=".true.">.false.</use_lch4>
 <use_nitrif_denitrif use_ed=".true.">.false.</use_nitrif_denitrif>
-<bgc_mode use_ed=".true.">ed</<bgc_mode>
+<bgc_mode use_ed=".true.">ed</bgc_mode>

 </namelist_defaults>

Even fixing that, your branch still has the same runtime error with decomp_depth_efolding described above.

[rgknox]

Hi @bandre-ucar,

I have a build here that I think is passing the restart read phase. Lawrencium has been very troublesome the last week due to IO problems, so its been hard to test. But I'm curious if the restart reads will work for you as well, and if there is any difference with how you implemented your fix.

https://github.com/bandre-ucar/ed-clm/pull/4

[ckoven]

@ekluzek, not sure how much background @bandre-ucar has given you on this, but the baseline tag that has the problem ought to be 853f006. I've been trying to test it via Ben's suggested fix of: ./create_test -testname ERS_D_Ld5.f10_f10.ICLM45ED.yellowstone_intel.clm-edTest -testid junk-r180 I'll paste my original email describing the problem to you below-- Thanks! Charlie

Erik,

I am trying to fix a scripting issue that has come up in the Ben’s recent merges of the FATES code to the CLM trunk. Basically the issue is that when use_ed is set to .true., the perl logic is somehow setting bgc_mode to “sp”. This doesn’t logically make sense, since ED ought to be mutually exclusive with SP. I think this used to work because ED was effectively a biophysics model only, but now that ED is connected to the soil biogeochemistry code this creates errors. So what I’d like is to define a new bgc_mode option of “ed”. so that whenever ed is on, that is the BGC mode as well.

I tried to do that in the xml file components/clm/bld/namelist_files/namelist_defaults_clm4_5.xml by adding the line:

ed

and then adding “ed” as a valid value for bgc_mode in components/clm/bld/namelist_files/namelist_definition_clm4_5.xml

and I also did the same in the testing scripts components/clm/bld/test_build_namelist/t/input/namelist_defaults_clm4_5_test.xml and components/clm/bld/test_build_namelist/t/input/namelist_definition_clm4_5_test.xml and

but none of this seems to work; the perl script is still receiving a value of “sp” in components/clm/bld/CLMBuildNamelist.pm. I don’t understand the syntax of the setup_cmdl_bgc subroutine in that script so I don’t get where that info actually comes from. Any chance you could help me get this sorted out?

Thanks, Charlie

[ekluzek]

@ckoven I have a branch on my NGEET fork that is basically functional. The clarification I want to make is that when ed is on, you also by default want the "bgc" settings right? So you get methane, nitrif-denitrif, century pools, and vertical carbon right? So you also need "use_cn" on, which also turns on some things you don't need like (lightning, and pop-density namelists). But, since we don't have switches for above and below ground processes, it seems that getting the "bgc" switches in addition to "use_ed=T" is what makes sense. I just want to confirm that's correct?

Later at some point we need a way to distinguish the above and below ground processes.

[ckoven]

@ekluzek thanks. Of the things you mention, we really only want methane, century pools, and vertical carbon when ED is on. Not the nitrif-denitrif option or the fang-fire-model-specific things (lightning, and pop-density) since ED doesn't have an N cycle yet and has its own fire model. I can't see your branch for whatever reason, but unless you changed the fortran, the nitrogen-cycle-specific things should still be off even though the bgc is flagged?

[ekluzek]

I haven't pushed my branch to the upstream main repository, so it's just on my fork right now. But, I'll try to get something that works as you've outlined and then push the branch upstream and let you know it's there.

And I'm just messing with the namelist generation.

Also I thought that methane requires nitrif_denitrif to be on?

[bandre-ucar]

As of 6ab0d89 the merge branch is up to date with clm-r181.

[bandre-ucar]

Testing of the c0654db changes into the r16x branch causes a restart failure in the coupler:

    FAIL ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_gnu.clm-edTest.cpl.hi.nc : test 
compare cpl.hi (.base and .rest files) 
    FAIL ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_gnu.clm-edTest : test functionality summary (ERS_test)

Failing fields from TestStatus.log

/glade/scratch/andre/ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_gnu.clm-edTest.07281644-edg/run/ERS_D_Ld5.f45_g37.ICLM45ED.yellowstone_gnu.clm-edTest.07281644-edg.cpl.hi.0001-01-06-00000.nc.rest.cprnc.out had the following fields that are NOT b4b  

   RMS l2x_Sl_anidr                     7.2491E-19            NORMALIZED  9.0283E-19
    RMS l2x_Sl_anidf                     7.2491E-19            NORMALIZED  8.8489E-19
    RMS l2x_Sl_tref                      4.4538E-15            NORMALIZED  1.6760E-17
    RMS l2x_Sl_qref                      5.2556E-18            NORMALIZED  1.0570E-15
    RMS l2x_Sl_t                         8.9077E-15            NORMALIZED  3.3675E-17
    RMS l2x_Sl_fv                        3.1606E-16            NORMALIZED  2.2130E-15
    RMS l2x_Sl_ram1                      2.2047E-13            NORMALIZED  4.5475E-16
    RMS l2x_Sl_u10                       2.4357E-15            NORMALIZED  8.2384E-16
    RMS l2x_Fall_swnet                   2.9692E-15            NORMALIZED  6.9590E-17
    RMS l2x_Fall_taux                    1.0439E-16            NORMALIZED  3.5435E-15
    RMS l2x_Fall_tauy                    1.0439E-16            NORMALIZED  3.5435E-15
    RMS l2x_Fall_lat                     3.6596E-13            NORMALIZED  3.0123E-14
    RMS l2x_Fall_sen                     1.8567E-13            NORMALIZED  1.3053E-14
    RMS l2x_Fall_lwup                    6.0869E-14            NORMALIZED  2.0968E-16
    RMS l2x_Fall_evap                    1.4619E-19            NORMALIZED  3.0530E-14
    RMS l2x_Fall_flxdst1                 7.3133E-25            NORMALIZED  6.5983E-16
    RMS l2x_Fall_flxdst2                 3.9265E-24            NORMALIZED  6.5999E-16
    RMS l2x_Fall_flxdst3                 9.2073E-24            NORMALIZED  6.5999E-16
    RMS l2x_Fall_flxdst4                 8.6713E-24            NORMALIZED  6.5986E-16

Note: all tests in the clm_short suite pass for yellowstone intel, gnu, pgi.

I poked at this a bit. Since the merged change incorporates three different bug fixes, I think the sanest way to debug the issues is going to be to back them out one at a time to narrow down the problem. For now I'm going to mark this test as an expected fail.

[rgknox]

In that last commit we modified the argument in the ice_mask to setFilters(). Yet, it looks like the second argument to setFilters is no longer the ice_mask and now is a "glc_behavior".

EDIT: I am trying to determine if we really need that call to setFilters (as some of the comments imply), that seems the most likely culprit to mess up something in the coupler.

[rgknox]

also, @bandre-ucar, I'm glad that we caught a new error, but where did the ERS_D_Ld5.f45_g37.ICLM45ED come from?

[bandre-ucar]

@rgknox The setFilters call was one of the conflicts, and I resolved it the same way you did in the prototype merge you made last week. I need to look more closely at it.