Underflows

[apcraig]

gfortran with debug flags is trapping several underflows in CICE. Some are known (ice_dyn_evp/eap) and others may not be (construct_fields in ice_transport_remap.F90). gfortran does not have an underflow to zero flag, and I'm not sure what happens when this arises without trapping. I assume it is underflowing to zero gracefully, but not sure. there are several things we could try. we could

• turn off underflow traps in gfortran debug
• code the underflow to zero as has been done in a few places in code already. some of this code is commented out due to cost concerns.
• there is supposed to be an ieee call to turn on underflow to zero at runtime (https://stackoverflow.com/questions/32851780/flush-to-zero-in-gfortran) but I have never tried this and I worry about portability (support with different compilers). We'd have to test carefully. This feature could also be turned on via a runtime or buildtime flag if needed.

We have coded underflow to zeros in icepack when needed without much regard to cost. But maybe that approach should be re-evaluated too so we bring some consistency to this issue. Mostly, we should continue to use a compiler's underflow to zero flag, and in those cases, we don't want to introduce a bunch of cost or complexity. See also https://github.com/CICE-Consortium/Icepack/issues/170

@eclare108213 @dabail10 @mattdturner any thoughts?

[apcraig]

Just did a quick test, if I turn of underflow traps with gnu in debug mode, the code runs to completion.

[apcraig]

The travis Macros file has underflow and invalid traps turned off right now. zero and overflow is turned on. On other machines with gnu, we are still trapping all 4 by default.

@anders-dc just wanted to add you to the discussion. Also, I notice there are differences in other compiler flags in travisCI_gnu and conrad_gnu (for instance). Should we consolidate towards a more consistent set of settings and what should they be?

[ghost]

Macros.travisCI_gnu is derived from the Icepack equivalent, which, in turn, was derived from the Macros.loft_gnu file. I think it is a good idea to carefully review all flags and establish a common base.

[mattdturner]

I think its ok to remove the check for underflows. While it is helpful sometimes to trap underflows, in my experience either flushing to 0 or allowing denormalized numbers is acceptable in most cases. The underflow check was likely initially included in GNU compile flags for debug since I initially tried to replicate the Intel compile flags. Intel -fpe0, though, automatically includes -ftz (flush to zero).

I think it is a good idea to carefully review all flags and establish a common base.

I agree

[apcraig]

@anders-dc @mattdturner I agree. There is a history with underflows in CICE. There was a time when they caused aborts and/or the underflows slowed the code down a lot. And there weren't always flush to zero compiler options. In recent years, we've been able to use a compiler flag, flush to zero, and that seems to work well and not slow the code down. Manual coding around underflows has been turned off, more or less. Now that we are doing more comprehensive testing, namely gnu with underflows trapped, we running into problems again.

I am starting to think that we should never trap for underflows with our debug mode anymore too. At least for compilers that don't have a flush to zero. I think the abort and performance issues are history and that we can rely on the compilers to do something reasonable. Does this also mean we should remove all code mods that work around underflows, even if currently commented out?

[apcraig]

I am looking at the gnu Macros files and am struggling a bit to figure out what compiler flags to use. Some of the current flags are g77 flags, some are gfortran flags, some are not documented at all that I can find. We have a couple different sets, the ones for conrad, onyx, and gordon. Then travisCI and loft are slightly different. Yet, I also don't see any complaints from the compiler about the flags. Maybe what I'll do is backoff to the most basic settings that compile and run and then we can move forward from that. I'm testing on conrad.