+Changing the ice ridging calls to Icepack

[kshedstrom]

• Icepack has made the call we were using private. The public interface we should have been using all along is the one here.
• Because icepack_step_ridge calls not just ridge_ice, but also cleanup_itd, answers change.
• cleanup_itd is called such that limit_aice is true. This will cause it to zap ice categories with very small areas. Will this clean up our ridging when using the new ridging schemes?
• Addresses issue #201, should be applied before updating Icepack.

[kshedstrom]

I should add that this isn't ready for prime time. I get a lot of these:

  (zap_snow_temperature)zap_snow_temperature: temperature out of bounds!
  (zap_snow_temperature)k:           1
  (zap_snow_temperature)zTsn:   158.039016468585
  (zap_snow_temperature)Tmin:  -100.000000000000
  (zap_snow_temperature)Tmax:  7.382919449499853E-008
  (zap_snow_temperature)zqsn:  -386044.334662961

I will investigate and see if I can fix it.

[kshedstrom]

We also need the resolution to https://github.com/CICE-Consortium/Icepack/issues/461

[marshallward]

After talking with @kshedstrom I agree that we should just stop using the ridge_ice call, since it was never meant to be called directly. Although this could potentially change answers, the number is likely to be very small (perhaps only Kate).

However, we should also probably wait until the issues with the new (correct?) solutions have been resolved before merging this.

There is the wider problem of calling Icepack directly inside of SIS2, rather than through a config_src wrapper, but we can address that later.

@kshedstrom I'll let you decide when you think this ought to go in.

[kshedstrom]

This one behaves for three hours with NEW_RIDGE_REDISTRIBUTION. Otherwise there are too many warnings. I'm going to try for a longer run next.

[kshedstrom]

This runs (slowly) without a mask_table, but dies with a mask_table:

forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source
libpthread-2.31.s  00007F7AC3472910  Unknown               Unknown  Unknown
libmpi_intel.so.1  00007F7AC476A3FF  Unknown               Unknown  Unknown
libmpi_intel.so.1  00007F7AC47A0AD6  Unknown               Unknown  Unknown
libmpi_intel.so.1  00007F7AC47A1896  MPI_Wait              Unknown  Unknown
libmpifort_intel.  00007F7AC644F93C  PMPI_WAIT             Unknown  Unknown
MOM6               0000000001BE4264  mpp_mod_mp_mpp_sy         224  mpp_util_mpi.inc
MOM6               0000000001747EEC  mpp_domains_mod_m         265  mpp_do_update.fh
MOM6               000000000174F985  mpp_domains_mod_m         166  mpp_update_domains2D.fh
MOM6               0000000000837EA0  mom_domain_infra_         208  MOM_domain_infra.F90
MOM6               000000000153FD13  sis_transport_mp_         339  SIS_transport.F90
MOM6               0000000001084E0F  sis_dyn_trans_mp_         625  SIS_dyn_trans.F90
MOM6               000000000089BD2D  ice_model_mod_mp_         324  ice_model.F90
MOM6               0000000000A22DDD  MAIN__                   1014  coupler_main.F90
MOM6               00000000004131DD  Unknown               Unknown  Unknown

There's a pass_var with a barrier there. Have you seen this before?

[kshedstrom]

This code is unacceptably slow. It must be happening inside cleanup_itd. I like the idea of having a cleanup_itd that gets rid of those negative ice thicknesses and concentrations in a mass and energy-conserving way, but not the way they do it.

[kshedstrom]

It turns out I do need the latest Icepack version. I was trying it out with 1.3.3, the last to work the old way.

[MJHarrison-GFDL]

@kshedstrom I am testing with v1.4 .

[MJHarrison-GFDL]

Getting many warnings. Diagnosing...

[kshedstrom]

This is all so confusing. I watched it work perfectly in the debugger on chinook (gnu). The runs that failed were on gaea (intel). It's still failing on gaea. My chinook job is "pending" - looks like the queue is being drained for a downtime.

[MJHarrison-GFDL]

With NEW_RIDGE_PARTICIPATION/REDISTRIBUTION = T I get a bunch of vice (ice volume) errors (~10^-3) and very large std output files. @kshedstrom , this might account for the model slow down.

[kshedstrom]

@MJHarrison-GFDL I had said

This one behaves for three hours with NEW_RIDGE_REDISTRIBUTION. Otherwise there are too many warnings.

My latest died after a few months with: WARNING from PE 113: Negative ice thickness at: 369 125 5 -1.8081E-46 This is with the check for negative ice thickness after the ridging. Is there anywhere else to get negative volumes?

[MJHarrison-GFDL]

This is an underflow value which should probably be reset to zero somwehere. Is this a case where Icepack is zapping small areas but leaving behind small negative residual volumes?

[kshedstrom]

I think Icepack is zapping both negative areas and volumes. This isn't coming from the ice_ridge check right after ridging, but somewhere else.

[MJHarrison-GFDL]

@kshedstrom I turned off the conservation check for ridging to avoid output warnings. Relative mass errors reported in seaice_stats is O(10^-13), which seems reasonable. Also got rid of the area_underflow check since itd_cleanup should take care of it. Two months and no problems reported. I will keep running to see if I get the same issue.

[MJHarrison-GFDL]

Fails in month 6 with underflow values for ice thickness using new ridging part/redist followed by a fatal in SIS_continuity.

WARNING from PE 366: Negative ice thickness at: 254 978 3 -8.7593E-60 FATAL from PE 366: Negative thickness input to ice_continuity()

[kshedstrom]

@MJHarrison-GFDL I too got this: WARNING from PE 159: Negative ice thickness at: 351 125 6 -4.2753E-48 FATAL from PE 159: Negative thickness input to ice_continuity().

I can turn on the mask_table, but only with some layouts. My 32x50 failed, but Yi-Cheng's 40x40 worked. I don't understand that one at all.