Decrease runtime by calibrating APCEMM thresholds

[Calebsakhtar]

Rough calibration of APCEMM parameters

Introduction

Issue https://github.com/MIT-LAE/APCEMM/issues/42 highlights a significant slowdown of APCEMM since commit https://github.com/MIT-LAE/APCEMM/commit/e1245bc6e9fbfd74d9cd71a9d24a454a85b47bf3 . This PR performs a rough calibration of the contrail threshold model parameter to improve computational performance whilst minimising loss of detail of the contrail sections. Please refer to the Appendix for full details of the calibration. The main text will only present a "before" and "after".

Final Recommendation

A recommendation to use a value of 0.90 for the contrail threshold is issued (formerly 0.01). This results in a reduction in runtime of ~80%.

Effects of changing from 0.01 to 0.90

Contrail runtime (1 thread)

The numbers below are approximate:

• 0.01 (default): 91min 17s
• 0.90: 19min and 23s
  • ~80% decrease in runtime relative to default

Contrail lifespan

Both the default case (0.01) and the calibrated case (0.90) persist for 16hrs at a timestep of 10 minutes.

Contrail properties

No significant differences in any of the section ice mass, ice particle number, or the integrated optical depth.

Contrail shape

Slight divergence in extinction-defined contrail shape parameters at the end of contrail life. However, this appears to be insignificant when considering that there are no observable changes in contrail properties.

Appendix

Contrail threshold calibration results

@sdeastham says: “In terms of what the contrail mask threshold means, it's subtle. You can think of it as a tracer chemical being left behind by the contrail ice which slowly diffuses out; once the concentration reaches X% (the threshold) of its initial value due to diffusion alone, the location is no longer included in the contrail mask. The idea is to allow areas which held water to continue being tracked after ice evaporates. However, the specific value to choose for the threshold is tricky to choose - and evidently important for reducing runtime! However, as the ice number threshold increases, the region in which the contrail tracer is set will shrink. The two factors therefore do work together somewhat. ### Contrail properties; effect on the section ice number (top) and on the ice mass (bottom):   No visible effect in either N or I. ### Contrail shape; effect on the section width (top) and on the depth (bottom):   No significant effect in either width or depth. ### Contrail optical properties: effect on the integrated optical depth:  No visible intOD divergence from any of the values. ### Runtimes - 1% (default): 91min 17s - 10%: 65min 51s - ~30% decrease in runtime relative to default - 30%: 49min 36s - ~45% decrease in runtime relative to default - 50%: 39min 15s - ~60% decrease in runtime relative to default - 70%: 27min and 14s - ~70% decrease in runtime relative to default - 80%: 21min and 8s - ~75% decrease in runtime relative to default - 90%: 19min and 23s - ~80% decrease in runtime relative to default ### Contrail lifespan - 1% (default): 16hrs - 10%: 16hrs - 30%: 15hrs 50 min - 10 min “premature” death - 50%: 16hrs - 70%: 16hrs - 80%: 15hrs 50 min - 0 min “premature” death - 90%: 16hrs ### Effect on Domain Size; default (top), 90% (bottom) sections at 6 hrs: Effective radius:   IWC:   Increasing the contrail threshold reduces the domain size significantly. This leads to the observed performance improvements. The contrail “shape” is determined by IWC distribution, which looks identical in both the increased contrail section case and the default case. It does however affect the distribution of ice particles, as seen by the change in r_eff distributions. This is also expected. ### Conclusions A “rough” calibration of the contrail threshold results in a recommendation of a value of 90% being chosen. No effects to the contrail properties are observed relative to the default value whilst achieving an 80% reduction in runtime. This suggests that that using a contrail mask might only be resulting in a slowdown of the code without much contribution.

(Not currently changed) Ice number threshold calibration results

@sdeastham says: “Locations which are considered to be "currently contrail containing" are defined by a new "contrail" tracer. Every time step, this is set to max(0,min(1,value + number * 1.0e6)). What that means in practice is that locations where the ice crystal number is greater than 1 particle per meter cubed are treated as "contrail containing" - an incredibly conservative assumption. You can change that factor on [this line](https://github.com/MIT-LAE/APCEMM/blob/67f95410b42753f24823c28fb8d2ae02aaf67dd4/Code.v05-00/src/Core/LAGRIDPlumeModel.cpp#L111). Specifically, if you change 1.0e6 to (say) 1.0, then the requirement is 1 particle per cm3; if it's 1.0e-6, then you need 1e6 per cm3, and so on. If locations contain no particles (or very few) then the contrail tracer is not deleted, but will diffuse over time (i.e. making sure you keep tracking the H2O).” ### Contrail properties; effect on the section ice number (first) and on the ice mass (second):   Note that the 1e3 line is almost coincident with the default line, while the 1e0 line diverges. ### Contrail shape; effect on the section width (first) and on the depth (second):   Like earlier, the 1e3 line remains close to the default, and the 1e6 line diverges greatly, ### Contrail optical properties: effect on the integrated optical depth:  No visible divergence from the 1e3 line. ### Runtimes: - 1e6 (default): 91min 17s - 1e3: 52min 56s - ~40% decrease in runtime relative to default - 1e0: 27min 42s - ~70% decrease in runtime relative to default ### Contrail Lifespan: - 1e6 (default): 16hrs - 1e3: 15hrs 50min - 10 min “premature” death - 1e0: 15hrs 40min - 20 min “premature” death ## Effect on Domain Size; default (top), 1e3 (bottom) sections at 6 hrs: Effective radius:   IWC:   Decreasing the ice threshold reduces the domain size significantly. This leads to the observed performance improvements. The contrail “shape” is determined by IWC distribution, which looks identical in both the reduced ice threshold case and the default case. It does however affect the distribution of ice particles, as seen by the change in r_eff distributions. This is also expected. ### Conclusions If the ice number threshold is to be changed by itself, a rough calibrated value for the ice number threshold of 1e3 is to be selected. This value minimally affects the integral ice particle number and the contrail ice mass. Runtime is decreased by 40% at a cost of a slight premature contrail death (when there barely is any ice left). No significant divergence in contrail physical, optical, and shape properties are observed when this value is selected.

[Calebsakhtar]

@sdeastham Would you mind replicating the results for the default case and the recommended case? Specifically, can you verify that the following:

• The performance improvement is of the same order of magnitude
• The contrail properties do not diverge

[sdeastham]

PR looks good. Test case shows 1.1% reduction in integrated mass over time, and a small decrease in lifetime. Run time on a single core decreased from 341 seconds to 120 seconds (65% reduction).