Investigation on single_step_call

minghangli-uni commented 7 months ago

Description

As was proposed in https://github.com/COSIMA/MOM6-CICE6/pull/48#discussion_r1558583441, there seems to be a discrepancy regarding the behavior of the single_step_call as described in the documentation and its actual implementation.

According to the documentation snippet provided here, single_step_call parameter is in default True, hence MOM advances its state with a single timestep, where DT_THERMO is neglected.

Issue

However, this is not the case. That is, when SINGLE_STEPPING_CALL=True & THERMO_SPANS_COUPLING=True, MOM does not advance with a single timestep!

Finding

On inspection of the code in src/core/MOM.F90, it appears that dt_therm is still in effect even when single_step_call is set to True. This observation is further confirmed by a sanity check via print outputs.
Additionally, it's noted that despite having different settings for single_step_call, the time consumption remains the same between two runs with thermo_does_span_coupling=True, dt=1350, and dt_thermo=2700. Both are around 25% faster than a comparison run with thermo_does_span_coupling=False
However, the diagnostics between the two runs are not the same.

I am currently investigating this issue as there seems to be some aspect that I am not yet aware of. This issue is considered critical as it could potentially affect the integrity of model solutions if left unresolved.

AndyHoggANU commented 7 months ago

Are you saying that thermo_does_span_coupling=True somehow overrides SINGLE_STEPPING_CALL=True?

minghangli-uni commented 7 months ago

After digging into the rabbit hole, it appears that we were misunderstanding the description of SINGLE_STEPPING_CALL,

  call get_param(param_file, mdl, "SINGLE_STEPPING_CALL", OS%single_step_call, &
                 "If true, advance the state of MOM with a single step "//&
                 "including both dynamics and thermodynamics.  If false, "//&
                 "the two phases are advanced with separate calls.", default=.true.)

SINGLE_STEPPING_CALL=True does not indicate this:

It's certainly confusing. Do we set single_step_call anywhere? It defaults to true, in which case DT, DT_THERM and THERMO_SPANS_COUPLING are ignored, according to this. Additional relevant documentation in MOM6 cap is here.

But both dynamics and thermodynamics advance together within a single subroutine call. Hence, DT_THERM, DT and THERMO_SPANS_COUPLING remain significant, and are preserved in this mode.

Conversely, whenSINGLE_STEPPING_CALL=False, it only indicates that dynamics and thermodynamics are stepping in two separate calls.

minghangli-uni commented 7 months ago

I am now investigating why diagnostics are not exactly the same when turning it on and off.

aekiss commented 6 months ago

This is still confusing to me. If dynamics and thermo are advanced in a single call with SINGLE_STEPPING_CALL=True, then DT_THERM and DT need to be the same, so dynamics and thermo advance by the same amount, right...?

minghangli-uni commented 6 months ago

Hi @aekiss From the code, unfortunately it seems not. As you may find here, dt_therm is updated within step_mom, whilst thermo_spans_coupling parameter continues to function within it.

After that, dt_therm is copied to dt_therm_here, here. Then step mom dynamics, where dt and dt_therm_here are not the same.

      call step_MOM_dynamics(forces, CS%p_surf_begin, CS%p_surf_end, dt, &
                             dt_therm_here, bbl_time_int, CS, &
                             Time_local, Waves=Waves)

aekiss commented 6 months ago

Ah, ok thanks for the clarification.

So single_step_call is only used to determine whether the NUOPC cap advances MOM over each coupling timestep with

(if true) a single call to step_MOM (with do_dynamics and do_thermodynamics defaulting to true), or
(if false) looping over pairs of calls to step_MOM (with one or the other of do_dynamics and do_thermodynamics set to false)

In both cases it looks (on a superficial skim) like within each coupling timestep the model loops over n_max dynamic and nts or ntstep thermodynamic steps, with this loop occurring either within the cap (if single_step_call is false) or within step_MOM (if single_step_call is true)... does that seem vaguely right?

minghangli-uni commented 6 months ago

Yes, I think you are right. From my understading, when dt=dt_cpl=dt_therm, results should remain consistent regardless of whether single_step_call is enabled or disabled. However, there seems to be a discrepancy in the results during field checks. Since I am now working on other things at the moment, I will revisit this issue later. But at the current stage, I think sticking with the default single_step_call=True seems like a prudent choice.

COSIMA / access-om3