Fix for exchange grid with add_gusts false: Only add gust fields if add_gusts is true

billsacks commented 2 months ago

Description of changes

Runs with aoflux_grid = "xgrid" and add_gusts = .false. have been failing with:

20240903 141813.539 ERROR            PET003 ESMCI_Container.h:178 ESMCI::Container::get() Invalid argument  - key does not exist: So_ugustOut

This PR fixes this issue by avoiding adding the two new gust fields to the list of mediator fields if add_gusts is false.

The explanation for why this caused an issue with the exchange grid is:

In mediator/esmFldsExchange_cesm_mod.F90: esmFldsExchange_cesm, So_ugustOut and So_u10withGust are added as atmosphere-ocean fluxes regardless of the value of add_gusts.
But in mediator/med_phases_aofluxes_mod.F90: set_aoflux_out_pointers, the fldbun_getfldptr call is only made if add_gusts is true.
There is a difference in behavior with xgrid than with ogrid here: with xgrid, the call to fldbun_getfldptr is responsible for adding the field to FBaof_x. (I think with ogrid or agrid, the fields have been added to the relevant field bundle already.)
So we end up with these gust fields not in FBaof_x, but still in fldnames_aof_out, and this leads to an inconsistency.

@megandevlan and @jedwards4b: I'm not positive that this is the right fix: It does solve the issue, but I'd like one or both of your input on whether this is the right way to solve the issue. In particular, can we rely on CAM working correctly when add_gusts is false if these fields are absent? It looks to me like CAM will set the relevant fields to 0 if they are absent; will that lead to correct behavior when add_gusts = .false.?

Specific notes

Contributors other than yourself, if any:

CMEPS Issues Fixed (include github issue #):

Are changes expected to change answers? No, but there are FIELDLIST diffs for cases with add_gusts false: the following fields will no longer be present on the cpl hi files when add_gusts is false:

Med_aoflux_ocn_So_u10withGust
Med_aoflux_ocn_So_ugustOut
Med_aoflux_atm_So_u10withGust
Med_aoflux_atm_So_ugustOut
atmExp_So_u10withGust
atmExp_So_ugustOut

Any User Interface Changes (namelist or namelist defaults changes)? No

Testing performed

Please describe the tests along with the target model and machine(s) If possible, please also added hashes that were used in the testing

Ran three versions of SMS_D_Ln9.ne30pg3_ne30pg3_mg17.FLTHIST.derecho_intel.cam-outfrq9s from cesm3_0_alpha03c:

add_gusts = .false., aoflux_grid = "ogrid"
add_gusts = .false., aoflux_grid = "xgrid" (This is the case that was failing previously.
add_gusts = .true., aoflux_grid = "xgrid"

All three pass now.

Also ran SMS_Ld40.TL319_t232.C_JRA.derecho_intel (which has add_gusts = .false.) with aoflux_grid = "xgrid"; this also passes now but failed before the changes in this PR.

I have also run the following tests with comparisons against baselines: all pass and are bit-for-bit, though with FIELDLIST diffs as expected:

SMS.TL319_t232.G_JRA.derecho_intel.mom-no_stoch_physics
ERP_Ln9.f09_f09_mg17.F1850.derecho_intel.cam-outfrq9s
SMS_Ld2.ne30pg3_t232.BLT1850.derecho_intel.allactive-defaultio with change to user_nl_cpl to set add_gusts false

billsacks commented 2 months ago

I have run a few additional tests with baseline comparisons and have updated the description above to note these tests.

megandevlan commented 2 months ago

Thanks @billsacks and @jedwards4b for getting to this so quickly! I believe that CAM should behave as expected when add_gusts = false, so this fix should be fine.

billsacks commented 2 months ago

Can you run a couple of tests using cam6 as well?

I got hung up in this testing due to #505 . But I have now done the following additional tests, using this diff:

diff --git a/cime_config/config_compsets.xml b/cime_config/config_compsets.xml
index 099602a..29b1333 100644
--- a/cime_config/config_compsets.xml
+++ b/cime_config/config_compsets.xml
@@ -38,6 +38,11 @@

   <!-- 1850 compsets Default, Mosart, Wave for CESM3 -->

+  <compset>
+    <alias>B1850</alias>
+    <lname>1850_CAM60_CLM50%SP_CICE_MOM6_MOSART_DGLC%NOEVOLVE_SWAV</lname>
+  </compset>
+
   <compset>
     <alias>BLT1850</alias>
     <lname>1850_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6_MOSART_DGLC%NOEVOLVE_SWAV</lname>

Ran these tests with CAM6:

SMS_Ld2.ne30pg3_t232.B1850.derecho_intel.allactive-defaultio (with default ogrid): passes and bit-for-bit
SMS_Ld2.ne30pg3_t232.B1850.derecho_intel.allactive-defaultio with aoflux_grid = "xgrid": passes (can't compare against baseline because the baseline fails as expected due to the issue fixed in this PR)

Note that I had also previously run ERP_Ln9.f09_f09_mg17.F1850.derecho_intel.cam-outfrq9s, which also uses CAM6; that also passed and was bit-for-bit with the baseline.

I have also run these additional tests with BLT1850 (which has add_gusts = .true.)

SMS_Ld2.ne30pg3_t232.BLT1850.derecho_intel.allactive-defaultio: passes and bit-for-bit
SMS_Ld2.ne30pg3_t232.BLT1850.derecho_intel.allactive-defaultio with aoflux_grid = "xgrid": passes but changed answers due to #505 (Update (2024-09-30): I redid a test to confirm that this branch is bit-for-bit once the fix in #506 is applied: with that fix applied to both the baseline and the new code, this branch is bit-for-bit with baseline)

@jedwards4b let me know if you feel this testing is sufficient; if so, I think this is ready to merge if you agree.

jedwards4b commented 2 months ago

Thanks for doing that test. I think that we are ready to merge.

ESCOMP / CMEPS