SW Development in the SW

[pernak18]

• [x] implement band fluxes as a cost function component (SW and LW) -- this can ostensibly be done right now, but the results may not be trustworthy
• [x] combine band k-distributions after optimization into a single, full k-distribution
• [x] include more forcings (SW and LW)
• [ ] weight g-point combinations with a scaling factor
• [x] end product for users with our cost function definition and normalization statistics so apples-to-apples comparison of reduced RRTMGP can be compared to other models that will produce similar RRTMGP flux netCDFs with their fluxes for Garand (or other profiles)
• [x] do something like the previous item for 1-angle vs. 3-angle RRTMGP calculations

[pernak18]

i've always suspected that the specious band fluxes were because of the executable and template netCDFs i was using, and i have confirmed this over the last few days. the rrtmgp_garand_atmos executable generated in this repository is only designed to work with single bands or broadband, not both (this has to do with how the flux variables are named in the templates -- band_flux_* is never used, only flux_* and the heating rate analogs). however, we do have an executable that does both. i have confirmed via profz and statz plotz in the LW (band 4 and BB) and SW (band 6 and BB) that this executable produces fluxes consistent with our baseline fluxes from the paper and Jen. the differences aren't exactly zero, but i do think they're small enough to blame different compilers, library versions, operating systems, etc.

since in the g-point reduction code, i use our executable for single bands, then stitch them together into a final flux file, band fluxes are accurate. broadband is then calculated from the accurate band fluxes. Eli has been working with broadband cost components and has given them his blessing, so everything is consistent now -- reduction has accurate by-band and broadband fluxes, and we have consistent, 256 g-point baseline fluxes in the bands and broadband.

in Slack DM

jend 6:47 PM You know - i think it is a bad idea to start with RRTMGP files that are populated with values at all for fluxes or heating rates - i am just realizing how badly this can go astray

response:

rick 9:07 AM @jend i haven't done an extensive look into this, but what you pointed out last night might explain those issues we were having a few weeks back of equal fluxes everywhere, equal BB HR, but different by-band HR when comparing your results to mine from a more recent RRTMGP run

[pernak18]

regarding additional weighting, from Fearless Leader (FL) in Slack DM:

trial i_a (current way) is k_comb = (wt_i k_i + wt_i+1 k_i+1) / (wt_i+wt_i+1) and trial i_b is k_comb = F (wt_i k_i + wt_i+1 * k_i+1) / (wt_i+wt_i+1), where F is some number like 0.995. so there would be twice as many trials as there are now

but we're holding off for now because (also from FL):

my little investigation comparing reference prof plots to iter93 prof plot has shown that the degradation of direct beam results is more in the stratosphere. strat delta-flux is greater in the higher number bands, due to rayleigh and strat o3 absorption, but the difference in strat delta-flux between the ref and iter93 is mostly in the lower number bands, which I would think is due to the very little solar absorption that occurs in the strat by co2. so i think i’ve concluded that I shouldn’t try to remedy this until I also have in place the co2 forcing results for the sw. perhaps just including that term in the CF might help this.

[pernak18]

1854c98c3a2719aaa92782460e9be960cf86bc01 addresses tasks 5 and 6 -- i.e., an end product that allows for cost function computation of other model applications (different models, modified RRTMGP [1- or 3-angle, e.g.], etc.) -- by pulling the cost calculation method out of its class and as a standalone function. this has the added benefit of parallelizing the computation for many g-point combination trials, which speeds up the process by a factor of ~8.

a driver script is still necessary to call this new function with different datasets and provide the user with some comparison metric (likely the normalized cost integrated over all weighted cost function components)

[pernak18]

5db2d373452113d8d7618de5158dabe1b6b934dc for comparing fluxes from reduction to other configurations like 1-angle (diffusivity and optimal) and 3-angle (full-k), etc.

[pernak18]

not sure if we're doing the remaining task -- FL has been reducing g-points in the LW and SW to his satisfaction